How to Solve Java Out of Memory Problem 2

[YerMom]

My Java app terminates unexpectedly with the java.lang.OutOfMemoryError. I'm going through my application trying to find possible causes of the problem, and I have a number of questions.

Question One
If I have a loop executes a great many times and the loop includes instructions like

String myString = "something";

Will this contribute to the out of memory error?

Question Two
My app has a number of data structures, and when I'm done with them I set them to null then call System.gc(). Is this the correct way to force garbage collection?

Question Three
One solution I'm trying is to increase the jvm heap size, but first I need to know what the default heap size is so I can start from there. How do I determine the default heap size?

Thanks!

 

[timw]

1. Any code which creates objects will naturally inflate the jvm process. Remember that using the literal "some string" form of creating a string is equivalent to doing a new String() instantiation.

2. The gc() method api documentation states that this doesn't necessarily foce garbage collection, it merely signals that you want one, and the jvm will comply at some point in the future. The jvm will aggressively collect the garbage when hard pressed for memory. Is it possible that there are live references to your data structures elsewhere in your code? This would prevent them from being gc'd.

3. I dunno what the default initial/maximum heap sizes are. I suspect it's differen between jvms on different platforms and I don't know of a way to make the jvm tell you. I believe, on Windows (32bit) the Sun JVM (1.4) has a default initial jvm size of 16Meg with a maximum of 64 meg. If you've got problems in your code such as live references keeping objects from being gc'd, then that needs tackling rather than just upping the max heap limit.

Tim

 

[YerMom]

Thanks timw. I'll examine my code for live references.

 

[stefanwagner]

I agree to timw's second statement, but would state it more assertive: You're discouraged from calling gc yourself, because you'll rarely know better than the JVM.

And before an OOME is thrown, GC is guaranteed to run anyway.

To question 1):
If you call String myString = new ... inside your loop, this String is approved for GC everytime the loop's end is reached, and the next iteration is taken.

Objects get free for GC if they leave their scope, i.e. where no further reference is reachable.
Therefore setting objects to null is allmost allways the wrong approach, and allways, if this is done just before the end of scope is reached.

void foo (Bar bar)
{
    String s = "foobar";
    bar.doSomething (s);
    // see remark below
    s = null;
}

The object s is referencing would get deleted here anyhow, if bar doesn't keep the copy, handed over by 'doSomething (s) ' - in which case s = null doesn't have any effect either.

don't visit my homepage:

http://home.arcor.de/hirnstrom/bewerbung

 

[YerMom]

Thanks stefan wagner.

I'm still experiencing the OOME problem, so apparently I've taken the wrong approach to solving the problem (assuming that bad memory management and poor program design is indeed the issue).

Can either of you give an example of what timw calls a live reference? I thought I understood the concept, but it looks like I don't.

My application opens up an XML file and constructs a list of objects from it. The objects represent the data in the XML file. Objects are nested, for example the "parent" object has Vectors of other objects, which in turn contain Vectors -- sometimes Strings, sometimes other objects. I don't finish using the list until the very end of the program. To make coding a little easier, each time I come to a new parent object in the list, I create a object from it:

for (int i = 0; i < parentObjectList.size(); i++)
{
   ParentObject p = parentObjectList.get(i);
   ChildObject1 = p.getChild1();
   // Do something with p and its nested objects. Examples:
   for (int j = 0; j<ChildObject1.size(); j++)
   {
      //Do something with ChildObject1
   }
   methodOne(p);
   methodTwo(p);
   // I'd like to show more details, but it's all
   // proprietary.
}

private void methodOne(ParentObject p)
{
  Vector<Child2> child2 = p.getChild2();
  //iterate through child2 and process its data.
  for (int i = 0; i < child2.size(); i++)
  {
    Vector<Child3> child3 = child2.get(i).getChild3();
    //do something with child3 Vector
}

Is this an OK way to approach the problem? Can you see potentials for memory leaks in this approach?

Thanks!

 

[timw]

It's hard to say whether your code is safe just be looking at what you post. If you are holding a full data structure in memory for the life of the program, and that structure has *lots* of nodes, then naturally if you don't allow the JVM enough memory it will OOME. I don't know if you're creating any extra unnecessary objects because though your posted code doesn't contain any explicit new statements, they could happen in the methods you call, or the code you don't post.

There is a tradeoff long acknowledged in computing .. to hold something entirely in memory and save processing time by avoiding having to reload nodes several times during a run, or to hold as little structure in memory as possible thus saving memory but take the hit in longer processing times due to potential reloads and object creation.



Tim

 

[YerMom]

Thanks, Tim.

No, in the methods I call from the primary for loop, I do not create any new objects with the new statement.

In other methods, I create additional objects (unrelated to the data structure in my previous post) with the new statement, but they all go out of scope when the methods terminate, so (according to what stefan wagner said)the objects then become eligible for garbage collection.

It's starting to look like my code is safe, and that the best solution is to increase the memory allotted to the jvm.

Do you know any web sites that provide additional information about Java memory management?

Thanks...

 

[timw]

Hmmm..

http://java.sun.com/j2se/reference/whitepapers/memorymanagement_whitepaper.pdf

http://www.javaworld.com/javaworld/javaqa/1999-08/04-qa-leaks.html

Tim

 

[stefanwagner]

Does it help to increase the memory?

Sometimes that solves your problem, and sometimes it just gives you some more seconds before crashing.
If your code produces recursive calls - parents being reinserted into their children -, increasing the memory will not help.

How big is your xml-file (bytes, lines, ...)?
How many objects do you expect beeing created?



don't visit my homepage:

http://home.arcor.de/hirnstrom/bewerbung

 

[YerMom]

Stefan,

Increasing memory to 128m solves the problem, although I do have some recursive calls.

The XML file size is 3,335,000 bytes and is 77743 lines.

I create about 580 objects.

If you need me to send you portions of my code I'd be glad to. I'd like to know for sure exactly what the problem is.

 

[stefanwagner]

That's about 50 characters per line, and more than 100 lines per object - do you throw most of the information away?

If 128m leads to a stable program - use it.

Searching for "java profiler" might help in investigating, where the memory goes.

don't visit my homepage:

http://home.arcor.de/hirnstrom/bewerbung