[IMAGING-355] Large animated GIF takes too much heap memory in getMetadata - ASF JIRA

Agile Board

Attach files

Attach Screenshot

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Bug
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: 1.0-alpha3
Fix Version/s: 1.0.0-alpha5
Component/s: Format: GIF
Labels:
None

Description

When calling ImageParser.getMetadata on large animated gif files, the java heap consumption is extremely high.

For example, see the test project I created:
https://github.com/andrewmkhoury/commons-imaging-gif-test

When calling ImageParser.getMetadata on the attached 5MB gif commons-imaging-test.gif, it uses ~1.5GB of heap space. When the max heap is set to -Xmx1488M or lower it fails with this exception. When the heap is set to -Xmx1489M it works.

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
	at java.base/java.util.Arrays.copyOf(Arrays.java:3745)
	at java.base/java.io.ByteArrayOutputStream.toByteArray(ByteArrayOutputStream.java:211)
	at org.apache.commons.imaging.mylzw.MyLzwDecompressor.decompress(MyLzwDecompressor.java:143)
	at org.apache.commons.imaging.formats.gif.GifImageParser.readImageDescriptor(GifImageParser.java:881)
	at org.apache.commons.imaging.formats.gif.GifImageParser.readBlocks(GifImageParser.java:596)
	at org.apache.commons.imaging.formats.gif.GifImageParser.readFile(GifImageParser.java:696)
	at org.apache.commons.imaging.formats.gif.GifImageParser.readFile(GifImageParser.java:680)
	at org.apache.commons.imaging.formats.gif.GifImageParser.getMetadata(GifImageParser.java:485)
	at org.apache.commons.imaging.formats.gif.GifImageParser.getMetadata(GifImageParser.java:58)
	at org.apache.commons.imaging.ImageParser.getMetadata(ImageParser.java:832)
	at Test.main(Test.java:28)

To generate the large gif file I did the following:
1. Install ffmpeg and gifsicle
2. Use quicktime to create a screen recording
3. Generate a gif out of the screen recording

ffmpeg -i ~/Desktop/Screen\ Recording\ 2023-06-29\ at\ 12.09.21\ PM.mov -pix_fmt rgb8 -r 10 commons-imaging-test.gif && gifsicle -O3 commons-imaging-test.gif -o commons-imaging-test.gif

To run the test program:

git clone git@github.com:andrewmkhoury/commons-imaging-gif-test.git
cd commons-imaging-gif-test
mvn assembly:assembly
java -Xmx1g -jar target/IMAGING-test-0.0.1-SNAPSHOT-jar-with-dependencies.jar commons-imaging-test.gif

Heap analysis via Eclipse MAT shows that the ImageDescriptor.imageData storing the bytes of each frame is the cause of the problem: