ifstream position pointer

[ParaToxic]

Hey ya, I have a question about the ifstream stuff. I would like to read a file with some threads.

I tried to read a file into 1 buffer but read in 2 pieces (first the first half and then the second).This was the first homework for me and now I want to fill 2 buffers (for example a vertex and a index buffer ) with 2 thread from one file.So that I read with one thread the first half of the file and put that in the vertex buffer and at the same time the second half into the index buffer.

 

BUT

 

I don't know how to set the position pointer to the half of the file, because you can only set the pointer to the begin,the current position and the end (file.seekg(0,ios::beg,cur,end); )

 

How can I do that ???

 

Thanks

[Canardia]

The 0 is the offset from either beginning or end. Just set it to length/2 if you want half of the file from the beginning:

 

is.seekg (0, ios::end); // goto end of file

length = is.tellg(); // get length of file

is.seekg (length/2, ios::beg); // goto half of file

 

http://www.cplusplus.../istream/seekg/

[ParaToxic]

Ahh thanks ,so easy

[Mumbles]

I want to fill 2 buffers (for example a vertex and a index buffer ) with 2 thread from one file.

 

You can only read from one location on the disk at a time. It may be a little nicer if you are using an SSD, but really, you would do just as well reading the first bit of data, then seeking to the second half of the file (using the above snippet) and reading the other bit of data, all in the same thread. This also doesn't waste CPU cycles switching between the threads.

 

A non-SSD drive would be a nightmare in that case if the chunks of data were at least 32 MB. Constantly jumping to different locations on the disk to serve the two threads at the same time, when disk I/O is the single slowest thing a computer can do... I/O is always better done in serial, rather than parallel.

[ParaToxic]

Hmmm you are right.I had a similar idea about that, but when you say that.

[Canardia]

I have tested different ways to read files, to find out which method is the fastest. As a result, I was able to write a fastcopy.exe which was much faster than Windows' own copy command

[ParaToxic]

Can you tell me how you did that ??

I found ot that ifstream is about 40% faster than FILE*

 

I can read a 67 MB File in 140 - 170 ms with ifstream , do you think it can be faster ?

[Canardia]

You can't compare them directly, and when you use FILE* correctly, it is much faster than ifstream. I think ifstream has a default buffer size which is is bigger than 1, and if you use FILE* and read always 1 byte at a time, then your FILE* is slower. To get FILE* to maximum speed, you need to analyze the media's cluster size and adjust your buffer size to that. There are more things to consider also, like what the cluster size of the harddisk is and/or how the raid was setup.

 

But a good default would be 16384, if you don't do any further media analysis to optimize the buffer size. Indeed, if you ever have setup raid disks, most OS will suggest a strip size of 16384, because it comes closest to the average file size on a file server, and the strip size should never be smaller than the cluster size on the harddisk.

[Mumbles]

When I first started reading files through the ifstream interface, I used to copy in blocks of 256 bytes. Obviously for huge files, it took forever.

 

Now I just seek to the end, get the overall length, make sure the pointer I'm reading it to is large enough, and then use that value as the number of bytes to read for the read command.

[Canardia]

I think that will cause ifstream internally to break it again down to some default size, which might be worse than 16384 (even bigger sizes can be slower, especially if they don't share a common divisor with the cluster size).