using io.SectionReader in Go

May 31, 2015

For my last task I was moving to an offset value in huge byte stream using reader.Seek(int64(offset), whence) and reading the bytes there. But it was not good for the performance because I was moving in a big file(~1gigs). Then I came to know about io.SectionReader and it made my life easier and helped me delete a lot of code(isn’t that best).

I create the section reader using this function:

func GetByteSection(pageBlock []byte, offset int64, sectionLength int64) *io.SectionReader {
    reader := bytes.NewReader(pageBlock)
    section := io.NewSectionReader(reader, offset, sectionLength)
    return section

Then I read it like:

tupleSection := GetByteSection(block, int64(headerOffset), int64(headerLength))
err = binary.Read(tupleSection, binary.LittleEndian, &data)
if err != nil {
    return nil, fmt.Errorf("failed to read t_bits: %v\n", err)

life has been easier with more sectionReader and less Seeks.