C++ fstream buffering is different than it was before

[jtm111]

I have been trying to understand this problem for about 6 months.

A few years ago in the old C++ one would use <fstream.h> which had a nice little function called setbuf().

ofstream myFile(&quot;source&quot;, ios::binary);
myFile.setbuf(buffer, size);
myFile.write(massivedata, int)...

which optimized I/O via a user-defined buffer. Trust me, when moving large amounts of data to files, it increased the I/O speed by a HUGE factor, often 10x as fast as a simple binary write. All my programs used it.

But C++ stream buffers are handled differently now with the new C++ classes defined in the file <fstream>. Here, an ofstream's setbuf() function is protected not public. I am told the way to set the buffer is to use the rdbuf() member and it via a public access function:

ofstream myFile(&quot;source&quot;, ios::binary);
myFile.rdbuf()->pubsetbuf(buffer, size);
myFile.write(massivedata, int)...

I'll tell you this, my friends: the new <fstream> class appears to be far slower than <fstream.h> in every possible way... in fact, using <fstream.h> without the optimization from setbuf() is twice as fast as the <fstream> class. Accessing rdbuf() through its public interface does nothing to the speed whatsoever.

I understand that the efficacy of buffering depends on experimenting with the size of the buffer; I have done this for several months and cannot replicate the speed I consistently get with <fstream.h>.

It is good programming discipline to use the most current C++ classes, so I would like to update to <fstream> rather than <fstream.h>.

Does anyone out there have any experience on this topic? Am I using fstream buffers wrongly?

 

[palbano]

Very interesting. I have to admit i have not used the old stream library since the stl has been around. What stl are you using VC 6, VC 7, STLPort ??

-pete

 

[jtm111]

palbano

I'm using VC6.

I use STL at every possibly opportunity and I am always impressed by its performance; the vector<> and map<> classes alone make it worth the trouble to learn about STL, iterators, and algorithms.

But I don't know offhand which STL classes I could use to perform file I/O. What would you use?

 

[palbano]

I have not had cause to be concerned with perfomance of file I.O. since my use of it is mostly limited to unit testing, logging, etc. I use ofstream when writing files.

You might try getting STLPort and seeing if there is a difference.

Keep us posted.
-pete

 

[jtm111]

I'll hunt around and let you know what happens. If I can get an STL solution, I would much prefer it. I'm really glad you brought it up because I wasn't even thinking in that direction. Thanks.