[IMAGING-355] Large animated GIF takes too much heap memory in getMetadata

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0-alpha3
• Fix Version/s: 1.0.0-alpha5
• Component/s: Format: GIF
• Assignee: Unassigned
• Reporter: Andrew Khoury
• Created: 29/Jun/23 22:06
• Updated: 02/Apr/24 18:51
• Resolved: 23/Dec/23 15:49

Description

When calling ImageParser.getMetadata on large animated gif files, the java heap consumption is extremely high.

For example, see the test project I created:
https://github.com/andrewmkhoury/commons-imaging-gif-test

When calling ImageParser.getMetadata on the attached 5MB gif commons-imaging-test.gif, it uses ~1.5GB of heap space.  When the max heap is set to -Xmx1488M or lower it fails with this exception. When the heap is set to -Xmx1489M it works.

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
	at java.base/java.util.Arrays.copyOf(Arrays.java:3745)
	at java.base/java.io.ByteArrayOutputStream.toByteArray(ByteArrayOutputStream.java:211)
	at org.apache.commons.imaging.mylzw.MyLzwDecompressor.decompress(MyLzwDecompressor.java:143)
	at org.apache.commons.imaging.formats.gif.GifImageParser.readImageDescriptor(GifImageParser.java:881)
	at org.apache.commons.imaging.formats.gif.GifImageParser.readBlocks(GifImageParser.java:596)
	at org.apache.commons.imaging.formats.gif.GifImageParser.readFile(GifImageParser.java:696)
	at org.apache.commons.imaging.formats.gif.GifImageParser.readFile(GifImageParser.java:680)
	at org.apache.commons.imaging.formats.gif.GifImageParser.getMetadata(GifImageParser.java:485)
	at org.apache.commons.imaging.formats.gif.GifImageParser.getMetadata(GifImageParser.java:58)
	at org.apache.commons.imaging.ImageParser.getMetadata(ImageParser.java:832)
	at Test.main(Test.java:28)

To generate the large gif file I did the following:
1. Install ffmpeg and gifsicle
2. Use quicktime to create a screen recording
3. Generate a gif out of the screen recording

ffmpeg -i ~/Desktop/Screen\ Recording\ 2023-06-29\ at\ 12.09.21\ PM.mov -pix_fmt rgb8 -r 10 commons-imaging-test.gif && gifsicle -O3 commons-imaging-test.gif -o commons-imaging-test.gif

To run the test program:

git clone [e-mail]:andrewmkhoury/commons-imaging-gif-test.git
cd commons-imaging-gif-test
mvn assembly:assembly
java -Xmx1g -jar target/IMAGING-test-0.0.1-SNAPSHOT-jar-with-dependencies.jar commons-imaging-test.gif

Heap analysis via Eclipse MAT shows that the ImageDescriptor.imageData storing the bytes of each frame is the cause of the problem:

Issue Links

• links to: GitHub Pull Request #301