Java OutOfMemoryError when saving RenderedImages 1

[oladis]

In an attempt to split a large image file, the program fails with the OutOfMemoryError. Further investigation revealed that this problem appears to be coming from the save routine (see below). When I expanded the upper limits of the JVM from the command line for example -Xmx1024m , the program runs a little longer before generating the same error. Unfortunately I never have enough memory to complete the splitting of the image file.

My question is how do I release/dispose the memory of a renderedImage or better still how do I make sure that the memory is released after each image is saved. I have tried System.gc but that has not solved my propblem. Any ideas will be greatly appreciated. By the way I am relatively new to Java so forgive me if this is appears to be a newbie question.

private void save(final RenderedImage image, final String filename)
throws FileAccessException
{

try
{
File saveFile = new File(filename);
ImageOutputStream out= ImageIO.createImageOutputStream(saveFile);
this.imageWriter.setOutput(out);
this.imageWriter.prepareWriteSequence(null);

IIOMetadata metadata = getMetadata(this.imageWriter, image, null);
IIOImage iioImage = new IIOImage(image, null, metadata);
this.imageWriter.writeToSequence(iioImage, this.imageWriterParam);
//
this.imageWriter.endWriteSequence();
//
out.close();
}
catch(IOException ioException)
{
throw new FileAccessException("Can not save the image file", ioException, "Contact system administrator");
}
}

 

[Diancecht]

Some ideas:

- Can you estimate the amount of memory you need? Maybe the problem is just that you need more memory, I'd try 2048.

- System.gc just tries to force garbage collection, but doesn't guarantee anything. And keep in mind that GC won't affect objects that are still referenced.

- Looking at your code, my only concern is what this.imagewriter is doing with the references. Are you calling save() more than once?

Cheers,
Dian

 

[kodr]

How large is your typical image file?

 

[oladis]

Thanks Diancecht but it is difficult to estimate how much memory I will need as the images can be of vaious sizes. The image that is causing the problem at the moment is about 92MB. I have tried the application with a much smaller file and it does work but and this suggests that more memory is required as the images pile up.

Assigning more memory is not an option as it is unable to reserve enough space for anything greater than 1200(see error below)

Error occurred during initialization of VM
Could not create the Java virtual machine.
Could not reserve enough space for object heap

Lastly, yes save is called multiple times (for each image). I have taken the liberty of pasting the enitire code hoping somebody can give me enough pointers to solve this problem.

Thanks in advance

public class WU1SplitImages
{
final static int TIFF_TAG_ORIENTATION = 274;
final static String TIFF_DIRECTORY = "tiff_directory";
final static int TIFF_TAG_XRESOLUTION = 282;
final static int TIFF_TAG_YRESOLUTION = 283;


private String imageFileName = null;
private String indexFilename = null;
private String outputDir = null;
private String outputIndexFilename = null;
private String outputFileNamePrefix = null;

private boolean hasMoreImageInCurFile = true;
private ImageDecoder imageDecoder = null;
private FileSeekableStream seekableStream = null;
private ParameterBlock paramBlock;
private TIFFDecodeParam decodeParam;

private ImageWriter imageWriter = null;
private ImageWriteParam imageWriterParam = null;
private long xResoultion = 0;
private long yResoultion = 0;

static int gidx = 0;
/**
* Creates a new instance of WU1SplitImages
*/
public WU1SplitImages()
{
}

/**
* The main function to split the image.
*/
void run() throws FileAccessException
{
// create output directory if not already exists
File dir = new File(this.outputDir);
if (!dir.exists())
{
dir.mkdirs();
}

// Gets tiff image writer
Iterator<ImageWriter> iterator = ImageIO.getImageWritersBySuffix("tiff");
this.imageWriter = iterator.next();

// Sets image writer params
this.imageWriterParam = this.imageWriter.getDefaultWriteParam();
if (this.imageWriterParam.canWriteCompressed())
{
this.imageWriterParam.setCompressionMode(ImageWriteParam.MODE_EXPLICIT);
this.imageWriterParam.setCompressionType("CCITT T.6");
this.imageWriterParam.setCompressionQuality((float)0.21); // C00001: quality ratio
}

this.imageDecoder = this.getImageDecoder(imageFileName);
this.outputFileNamePrefix = this.outputDir + "\\";
WU1IndexFile indexFile = new WU1IndexFile(indexFilename, outputIndexFilename);

int index = 0;
String frontImgFilename = null;
String backImgFilename = null;

RenderedOp image = null;
//RenderedImage image = null;
while(this.hasMoreImageInCurFile)
{
image = getRenderedOp(this.paramBlock, this.decodeParam);

if ((++index % 2) == 1)
{
frontImgFilename = "Image_" + index + ".tif";
this.processImage(image, outputFileNamePrefix + frontImgFilename);
}
else
{
backImgFilename = "Image_" + index + ".tif";
this.processImage(image, outputFileNamePrefix + backImgFilename);
indexFile.updateRecord(frontImgFilename, backImgFilename);
}
if (image != null)
{
image.dispose();
image = null;
}

}

indexFile.close();
}

/**
* Processes the specific image.
*
* @param image the image to be processed.
* @param outputImageFilename output image file name.
*
* @throws IOException if IOException occurs.
*/
private void processImage(RenderedImage image, String outputImageFilename)
throws FileAccessException
{
this.processResolution(image);

this.setCompressType(image); //C00001

this.save(image, outputImageFilename); //C00001
//final RenderedOp transposedImage = this.transposeImage(image);
//this.disposeImage(transposedImage); // C00001

}

/**
* Gets the resoultion of the specific image.
*/
private void processResolution(final RenderedImage image)
{
TIFFDirectory tiffDir=(TIFFDirectory)image.getProperty(TIFF_DIRECTORY);

this.xResoultion = this.getResolution(tiffDir, TIFF_TAG_XRESOLUTION);
this.yResoultion = this.getResolution(tiffDir, TIFF_TAG_YRESOLUTION);
}

/**
* Gets the specfic tag value of the specific TIFFDirectory.
*/
private long getResolution(final TIFFDirectory tiffDirectory, final int tag)
{
final TIFFField tiffResolutionField = tiffDirectory.getField(tag);

final long[] resolutions = tiffResolutionField.getAsRational(0);

return resolutions[0];
}

/** C00001 Start
* set compression type for each image
*/
private void setCompressType(final RenderedImage image)
throws FileAccessException
{
TIFFDirectory tiffDir = (TIFFDirectory)image.getProperty(TIFF_DIRECTORY);
TIFFField tiffOrientationField = tiffDir.getField(TIFF_TAG_ORIENTATION);

TIFFField tiffCompressionField = tiffDir.getField(BaselineTIFFTagSet.TAG_COMPRESSION);
int compression = tiffCompressionField.getAsInt(0);
if (compression == BaselineTIFFTagSet.COMPRESSION_JPEG)
{
// set writer compression type JPEG;
this.imageWriterParam.setCompressionType("JPEG");
}else if (compression == BaselineTIFFTagSet.COMPRESSION_CCITT_T_6)
{
// set writer compression type ;
this.imageWriterParam.setCompressionType("CCITT T.6");
}else
{
throw new FileAccessException("The image has unexpected " +
"Compression: " + compression, "", "Contact system " +
"administrator");
}

}
// C00001 End

/**
* Saves the specfic image to a specific file.
*/
private void save(final RenderedImage image, final String filename)
throws FileAccessException
{

try
{
File saveFile = new File(filename);
ImageOutputStream out= ImageIO.createImageOutputStream(saveFile);
this.imageWriter.setOutput(out);
this.imageWriter.prepareWriteSequence(null);

IIOMetadata metadata = getMetadata(this.imageWriter, image, null);
IIOImage iioImage = new IIOImage(image, null, metadata);
this.imageWriter.writeToSequence(iioImage, this.imageWriterParam);
//
this.imageWriter.endWriteSequence();
//
out.close();

if (++gidx == 1000)
{
System.gc();
}
}
catch(IOException ioException)
{
throw new FileAccessException("Can not save the image file",
ioException, "Contact system administrator");
}
}

/**
* Gets the meta data of a specific image.
*/
private IIOMetadata getMetadata(ImageWriter imageWriter, RenderedImage
image, ImageWriteParam param)
{
TIFFImageMetadata tiffMetadata = (TIFFImageMetadata) getIIOMetadata(image,
imageWriter, param);
TIFFIFD rootIFD = tiffMetadata.getRootIFD();
BaselineTIFFTagSet base = BaselineTIFFTagSet.getInstance();

// generate only 1 stripe: set RowsPerStrip to 2^16-1
rootIFD.addTIFFField(new com.sun.media.imageio.plugins.tiff.TIFFField(
base.getTag(278), 65535));

// prepare array for DPI values (dpi/1)
long xResolution[][] = new long[1][2];
xResolution[0] = new long[] {this.xResoultion, 1};
rootIFD.addTIFFField(new com.sun.media.imageio.plugins.tiff.TIFFField(
base.getTag(282), 5, 1, xResolution));

long yResolution[][] = new long[1][2];
yResolution[0] = new long[] {this.yResoultion, 1};
rootIFD.addTIFFField(new com.sun.media.imageio.plugins.tiff.TIFFField(
base.getTag(283), 5, 1, yResolution));

// C00001 add resolutionUnit tag: inch
rootIFD.addTIFFField(new com.sun.media.imageio.plugins.tiff.TIFFField(
base.getTag(296), 3 , 1, (Object) new char[] {2}));

return tiffMetadata;
}

/**
* Gets IIO meta data of the specific image.
*/
private IIOMetadata getIIOMetadata(RenderedImage image, ImageWriter
imageWriter, ImageWriteParam param)
{
ImageTypeSpecifier spec = ImageTypeSpecifier.createFromRenderedImage(
image);
IIOMetadata metadata = imageWriter.getDefaultImageMetadata(spec, param);

return metadata;
}

/**
* Gets the image decoder of a specific file.
*/
private ImageDecoder getImageDecoder(final String filename)
throws FileAccessException
{
try
{
final File imageFile = new File(filename);

if (imageFile.exists())
{
this.seekableStream = new FileSeekableStream(imageFile);

this.paramBlock = new ParameterBlock();
this.decodeParam = new TIFFDecodeParam();
this.paramBlock.add(this.seekableStream);
this.paramBlock.add(this.decodeParam);

return ImageCodec.createImageDecoder("tiff", seekableStream,
null);
}
else
{
throw new FileAccessException(
"File does not exist: " + filename, "",
"Check the file name and path.");
}
}
catch (IOException ioException)
{
throw new FileAccessException(
"Can not open file : " + filename, ioException,
"Check the file permission.");
}
}

/**
* Gets a RenderedOp from a specific ParameterBlock.
*/
private RenderedOp getRenderedOp(ParameterBlock paramBlock,
TIFFDecodeParam decodeParam) throws FileAccessException
{
RenderedOp renderedOp = null;

if(!this.hasMoreImageInCurFile)
{
throw new FileAccessException("Unexpected end of multi-tiff " +
"file", "Invalid image file.", "Contact system administrator");
}
else
{
renderedOp = JAI.create("tiff", paramBlock);
TIFFDirectory dir = (TIFFDirectory)renderedOp.getProperty(
TIFF_DIRECTORY);
long nextOffset = dir.getNextIFDOffset();

if (nextOffset != 0)
{
decodeParam.setIFDOffset(nextOffset);
}
else
{
this.hasMoreImageInCurFile = false;
}
}

return renderedOp;
}

/**
* Disposes the specific image.
*
* @param image the image to be disposed.
*/
private void disposeImage(RenderedOp image)
{
if (image != null)
{
image.dispose();
image = null;
}
}

/**
* Gets the configuration parameters.
*
* @param properties the properties, can not be null.
*/
public void getConfigParameters(final Properties properties)
throws IOException
{
this.imageFileName =properties.getProperty("Input.ImageFilename");
this.indexFilename =properties.getProperty("Input.IndexFilename");
this.outputDir = properties.getProperty("Output.ImageDirectory");
this.outputIndexFilename = properties.getProperty("Output.IndexFilename");
}

private RenderedImage getRenderedImage(ParameterBlock paramBlock,
TIFFDecodeParam decodeParam) throws FileAccessException
{
RenderedImage renderedImage = null;

if(!this.hasMoreImageInCurFile)
{
throw new FileAccessException("Unexpected end of multi-tiff " +
"file", "Invalid image file.", "Contact system administrator");
}
else
{
renderedImage = JAI.create("tiff", paramBlock);
TIFFDirectory dir = (TIFFDirectory)renderedImage.getProperty(
TIFF_DIRECTORY);
long nextOffset = dir.getNextIFDOffset();

if (nextOffset != 0)
{
decodeParam.setIFDOffset(nextOffset);
}
else
{
this.hasMoreImageInCurFile = false;
}
}

return renderedImage;
}


}

 

[oladis]

Hi Kodr - Typically between 60 - 90 MB.
Should I also mention that the images recently changed from black and white to grey scale and that is when all these issues started.

 

[kodr]

   private ImageDecoder imageDecoder = null;
   private FileSeekableStream seekableStream = null;
   private ParameterBlock paramBlock;
   private TIFFDecodeParam decodeParam;
   
   private ImageWriter imageWriter = null;
   private ImageWriteParam imageWriterParam = null;

Do all of these need to be declared at the class level? Maybe try declaring them in a more specific scope if possible.

Also, (and I doubt this is a good practice) you can put Runtime.getRuntime().freeMemory() to determine how much memory is still available to you. Possible place that in your main loop and see how much memory is being used up between it iteration.

 

[Diancecht]

Appart from kodr's good advice, how are you setting the heap size?

It's a good practice to set bot Xmx for max size and Xms for initial size (let's say 512Mb) so maybe you can get rid of that error.

Which OS are you using?

Cheers,
Dian

 

[oladis]

I do set the initial and max heap size but I only have a maximum of 1024 available. I am on windows XP. I have also found out that there appears to be an issue with the image writer API relating to memory leaks which does not appear to have been resolved. I might have to investigate other alternatives at this point.