Buffering Reads with bytes.Buffer

One handy tool handy tool that I reach for time and time again is bytes.Buffer (see the package docs).

Say you have an object that implements io.Reader (that is, it has a method with the signature Read([]byte) (int, os.Error)).

A common example is an os.File:

f, err := os.Open("/etc/passwd", os.O_RDONLY, 0644)

If you wanted to read the contents of that file into a string, simply create a bytes.Buffer (its zero-value is a ready-to-use buffer, so you don’t need to call a constructor):

var b bytes.Buffer

Use io.Copy to copy the file’s contents into the buffer:

n, err := io.Copy(b, f)

(An alternative to using io.Copy would be b.ReadFrom(f) – they’re more or less the same.)

And call the buffer’s String method to retrieve the buffer’s contents as a string:

s := b.String()

The bytes.Buffer will automatically grow to store the contents of the file, so you don’t need to worry about allocating and growing byte slices, etc.

31825 views and 3 responses

  • Aug 13 2010, 1:58 AM
    hokapokadotcom (Twitter) responded:
    Hi Andrew

    I really like your examples, very clear & also very helpful.

    I hadn't realized that bytes.Buffer returned a zeroed buffer ready to use, I've been calling the constructor every time I've used it. - Time to go through and change them.

    Also, where I've been reading the *http.Response.Body, that implements io.Reader, I have used b, _ := ioutil.ReadAll( r.Body ), of course b here is a bytes[], and then into a string via s := string(b)

    Would it be more efficient to use the copy method than what I have used with ioutil.ReadAll ?

    Your approach feels more appropriate or more compatible with Go idioms.

    Many thanks for the great examples.

  • Aug 13 2010, 7:41 AM
    Andrew Gerrand responded:
    If you look at the implementation of ioutil.ReadAll (http://golang.org/src/pkg/io/ioutil/ioutil.go#L17) you can see that it very simply uses a bytes.Buffer in much the same way I've used it here. Feel free to keep using ioutil.ReadAll - that's what it's there for, after all. Bytes.Buffer might offer you more flexibility if you want to do more than just take a []byte - for example, bytes.Buffer implements the io.Reader interface.

    Glad you're enjoying the blog posts! :-)

  • Aug 21 2010, 4:07 PM
    Yves Junqueira responded:
    Thanks for the useful posts, Andrew. This one made me go back and rewrite a few things ;-).
    (e.g: UDP socket reads, although I'm still not getting them right) .