read data from file... 1

[DotNetGnat]

Guys,

I need to create a double link list off of the data from a text file. I do not want code for reading from a file. I just need suggestions of efficient of doing this task assuming that have millions of data points...

Efficient in the sense...efficient use of memory...no memory leaks and so on...

any suggestions...

thanks in advance

-DNG

 

[Salem]

http://www.sgi.com/tech/stl/List.html

--
If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.

 

[DotNetGnat]

Any other way of achieving efficiency without using list...just conceptually...

thanks

-DNG

 

[Salem]

Such as?

The efficiency which comes with using well tested approaches so you can focus on the the problem at hand, rather than worrying about minutiae.

I mean, using the STL you could bang out the code in a few minutes and be done with it.

Or you could spend days hand-crafting something special for this particular case, perhaps weeks chasing obscure bugs caused by your overly-clever approach. At the end of the day (or rather many days), you've saved 10% of memory, and perhaps made it 5% faster.
Is that more efficient? The people paying your salary might disagree.

Or if the program takes an hour to run, is there any point in making it run in 55 minutes?
People will just set it to run before lunch, but they won't be in any hurry to rush back to see how quick it was this time.

> I just need suggestions of efficient of doing this task assuming that have millions of data points...
You can only assess performance in a total program. What seemed like a pretty smart idea locally could have disasterous implications elsewhere.

For example, did you choose 'list' because you're worried about the cost of expanding an 'array' or a 'vector'?

Rule 1: Make it right, before you try to make it fast.
Rule 2: Measure first, then improve, then measure again.

> I do not want code for reading from a file
See, there's a problem already.
Memory access times are measured in units of 1E-9 seconds.
Disk seek times are measured in units of 1E-3 seconds.

Consider two simple tests

list<int> L;
clock_t now = clock();
for ( int i = 0 ; i < 1E7 ; i++ ) L.push_back(i);   // fill the list, nothing more
cout << "Time taken = " << clock() - now << endl;
cout << "Last element = " << L.back() << endl;

ifstream fin("myfile.txt");
int myInt;
clock_t now = clock();
while ( fin >> myInt ); // read the file, nothing more
cout << "Time taken = " << clock() - now << endl;
cout << "Last element = " << myInt << endl;

Say the first takes 5 seconds, and the second takes a minute. It really doesn't matter how much you optimise the first, you're stuck with the second as being the elephant in the room.


--
If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.

 

[DotNetGnat]

Thanks Salem,

Nice tips...I really appreciate it...Have a star...

-DNG

 

[uolj]

As Salem hinted, a vector is often faster than a list in practice. I'd consider starting with that, then optimizing if necessary after you've done some testing.

If you design your code well, then the impact of switching from vector to list (or to other solutions) shouldn't be too difficult.