Buffering Reads with bytes.Buffer

August 12 2010, 8:57 PM  by Andrew Gerrand

One handy tool handy tool that I reach for time and time again is bytes.Buffer (see the package docs).

Say you have an object that implements io.Reader (that is, it has a method with the signature Read([]byte) (int, os.Error)).

A common example is an os.File:

f, err := os.Open("/etc/passwd", os.O_RDONLY, 0644)

If you wanted to read the contents of that file into a string, simply create a bytes.Buffer (its zero-value is a ready-to-use buffer, so you don’t need to call a constructor):

var b bytes.Buffer

Use io.Copy to copy the file’s contents into the buffer:

n, err := io.Copy(b, f)

(An alternative to using io.Copy would be b.ReadFrom(f) – they’re more or less the same.)

And call the buffer’s String method to retrieve the buffer’s contents as a string:

s := b.String()

The bytes.Buffer will automatically grow to store the contents of the file, so you don’t need to worry about allocating and growing byte slices, etc.

Tags

code, go, stackoverflow

31825 views and 3 responses

  • Aug 13 2010, 1:58 AM
    hokapokadotcom (Twitter) responded:
    Hi Andrew

    I really like your examples, very clear & also very helpful.

    I hadn't realized that bytes.Buffer returned a zeroed buffer ready to use, I've been calling the constructor every time I've used it. - Time to go through and change them.

    Also, where I've been reading the *http.Response.Body, that implements io.Reader, I have used b, _ := ioutil.ReadAll( r.Body ), of course b here is a bytes[], and then into a string via s := string(b)

    Would it be more efficient to use the copy method than what I have used with ioutil.ReadAll ?

    Your approach feels more appropriate or more compatible with Go idioms.

    Many thanks for the great examples.

  • Aug 13 2010, 7:41 AM
    Andrew Gerrand responded:
    If you look at the implementation of ioutil.ReadAll (http://golang.org/src/pkg/io/ioutil/ioutil.go#L17) you can see that it very simply uses a bytes.Buffer in much the same way I've used it here. Feel free to keep using ioutil.ReadAll - that's what it's there for, after all. Bytes.Buffer might offer you more flexibility if you want to do more than just take a []byte - for example, bytes.Buffer implements the io.Reader interface.

    Glad you're enjoying the blog posts! :-)

  • Aug 21 2010, 4:07 PM
    Yves Junqueira responded:
    Thanks for the useful posts, Andrew. This one made me go back and rewrite a few things ;-).
    (e.g: UDP socket reads, although I'm still not getting them right) .