Buffering Reads with bytes.Buffer
One handy tool handy tool that I reach for time and time again is
bytes.Buffer (see the package docs).
Say you have an object that implements
io.Reader (that is, it has a
method with the signature
Read(byte) (int, os.Error)).
A common example is an
f, err := os.Open("/etc/passwd", os.O_RDONLY, 0644)
If you wanted to read the contents of that file into a string, simply
bytes.Buffer (its zero-value is a ready-to-use buffer, so
you don’t need to call a constructor):
var b bytes.Buffer
io.Copy to copy the file’s contents into the buffer:
n, err := io.Copy(b, f)
(An alternative to using
io.Copy would be
b.ReadFrom(f) – they’re
more or less the same.)
And call the buffer’s String method to retrieve the buffer’s contents as a string:
s := b.String()
bytes.Buffer will automatically grow to store the contents of
the file, so you don’t need to worry about allocating and growing
byte slices, etc.
31825 views and 3 responses
Aug 13 2010, 1:58 AMhokapokadotcom (Twitter) responded:Hi Andrew
I really like your examples, very clear & also very helpful.
I hadn't realized that bytes.Buffer returned a zeroed buffer ready to use, I've been calling the constructor every time I've used it. - Time to go through and change them.
Also, where I've been reading the *http.Response.Body, that implements io.Reader, I have used b, _ := ioutil.ReadAll( r.Body ), of course b here is a bytes, and then into a string via s := string(b)
Would it be more efficient to use the copy method than what I have used with ioutil.ReadAll ?
Your approach feels more appropriate or more compatible with Go idioms.
Many thanks for the great examples.
Aug 13 2010, 7:41 AMAndrew Gerrand responded:If you look at the implementation of ioutil.ReadAll (http://golang.org/src/pkg/io/ioutil/ioutil.go#L17) you can see that it very simply uses a bytes.Buffer in much the same way I've used it here. Feel free to keep using ioutil.ReadAll - that's what it's there for, after all. Bytes.Buffer might offer you more flexibility if you want to do more than just take a byte - for example, bytes.Buffer implements the io.Reader interface.
Glad you're enjoying the blog posts! :-)
Aug 21 2010, 4:07 PMYves Junqueira responded:Thanks for the useful posts, Andrew. This one made me go back and rewrite a few things ;-).
(e.g: UDP socket reads, although I'm still not getting them right) .