Vector of arrays balloons memory 1

[JimmyFo]

Hi, using this basic setup:

struct frame

{
int frameNmbr[153];
string phone[153];
int state[153];
float logP[153
};


infile3.open("output.txt");

i=j=0;
frame newestFrame;

for(i=0; i < 558; i++)
{
for(j=0; j < 153; j++)
{
infile3>>newestFrame.frameNmbr[j];
infile3>>newestFrame.phone[j];
infile3>>newestFrame.state[j];
infile3>>newestFrame.logP[j];
arrOut.push_back(newestFrame);
}
}

infile3.close();


Running just that, the memory usage balloons to nearly 500 MB Ram and still hangs for a minute or two every 100 iterations of i. Why is this?

Thanks,
James

 

[uolj]

You have two nested loops, one running 558 times and another 153 times. Each time through the inner loop you add a frame to the vector. That means you are adding 85374 frame objects into the vector.

Each frame object has 306 ints, 153 floats, and 153 strings. On my machine, that's 306*4 bytes + 153*4bytes + 153*28 bytes since an int and float are 4 bytes and a string is 28. That gives 6120 bytes per frame object.

So the total memory you are adding to the vector is 85374*6120 bytes which is 522488880 bytes, or about 500MB.

Maybe you meant to put the call to push_back outside the inner loop?

 

[JimmyFo]

Wow, I never thought of it that way. that's what I want to do, but I'll have to pull it apart and do it another way. Thanks for your help!

 

[uolj]

Are you sure you don't need to just move the push_back call down two lines? It will reduce the memory consumption to about 3MB instead of 500, and since your index j goes to 153 I would guess that's really what you meant to do.

 

[JimmyFo]

You're right. I have 558 frames of 4x153 pieces of information. That will do the trick. Thank you!

 

[uolj]

By the way, since you are using vector with push_back and you know how many objects you will add, you should call reserve to allocate the memory ahead of time. This should help your performance by eliminating the need for the vector to resize itself as you add more objects, which leads to excess allocating, deallocating and copying.

Assuming arrOut is empty to start with, put this line before the for loops:

arrOut.reserve(558);

 

[JimmyFo]

Great stuff! Go for TipMaster