Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Later
    • Affects Version/s: 0.94-incubator
    • Fix Version/s: None
    • Component/s: Format: JPEG
    • Labels:
      None
    • Environment:

      win32, 32 bit operating systems

      Description

      This function can throw an Exception in ByteSourceArray.java due to a negative byte[] allocation size. The length argument has been found to wrap when called from IccProfileParser.java.

      In 64bit machines, issues related to incorrect metadata, or ICC data can lead to incorrect and excess memory allocations. These large numbers however cause 32bit negative signed values.

      public byte[] getBlock(int start, int length) throws IOException

      { if (start + length > bytes.length) throw new IOException("Could not read block (block start: " + start + ", block length: " + length + ", data length: " + bytes.length + ")."); byte result[] = new byte[length]; System.arraycopy(bytes, start, result, 0, length); return result; }
      1. concat-app13.patch
        3 kB
        Damjan Jovanovic
      2. crash.jpeg
        95 kB
        Brien Voorhees

        Activity

        Hide
        Damjan Jovanovic added a comment -

        Deferring to after the 1.0 release.

        Show
        Damjan Jovanovic added a comment - Deferring to after the 1.0 release.
        Hide
        Damjan Jovanovic added a comment -

        Concatenating the APP13 segments, as this patch does, gets the metadata to parse without an exception.

        However this alone is not the correct solution, as some images contain independent APP13 blocks which should not be concatenated. Detecting which blocks belong together is a difficult problem, and http://dev.exiv2.org/issues/533 describes how exiv2 does it (by parsing the internal block structures).

        Show
        Damjan Jovanovic added a comment - Concatenating the APP13 segments, as this patch does, gets the metadata to parse without an exception. However this alone is not the correct solution, as some images contain independent APP13 blocks which should not be concatenated. Detecting which blocks belong together is a difficult problem, and http://dev.exiv2.org/issues/533 describes how exiv2 does it (by parsing the internal block structures).
        Hide
        Damjan Jovanovic added a comment -

        EXIF was changed to silently ignore bad tags when not in strict mode - exceptions are only thrown in strict mode. It seems like a good idea to add this for IPTC as well.

        That image has multiple APP13 segments. Even if we break instead of throw when the invalid block size happens, there is another exception later because Sanselan doesn't handle multiple APP13 segments. Even if only the first segment is returned there, it has no useful data.

        On the other hand, there is some debate (http://dev.robotbattle.com/bugs/view.php?id=533 and http://uk.groups.yahoo.com/group/exiv2/message/1382) as to whether files with multiple APP13 segments are valid, because supposedly the APP13 segments can be joined together in some cases. Maybe when joined together than now invalid length of 86622 becomes valid?

        Show
        Damjan Jovanovic added a comment - EXIF was changed to silently ignore bad tags when not in strict mode - exceptions are only thrown in strict mode. It seems like a good idea to add this for IPTC as well. That image has multiple APP13 segments. Even if we break instead of throw when the invalid block size happens, there is another exception later because Sanselan doesn't handle multiple APP13 segments. Even if only the first segment is returned there, it has no useful data. On the other hand, there is some debate ( http://dev.robotbattle.com/bugs/view.php?id=533 and http://uk.groups.yahoo.com/group/exiv2/message/1382 ) as to whether files with multiple APP13 segments are valid, because supposedly the APP13 segments can be joined together in some cases. Maybe when joined together than now invalid length of 86622 becomes valid?
        Hide
        Brien Voorhees added a comment -

        Thanks for looking into the issue. I actually did attach the image
        ('crash.jpeg') when I posted my comment. It appears to still be there
        (at the top of the page under "Image Attachments"). Just let me know if
        that doesn't work.

        I implemented a quick work around for the issue. Not sure if it's
        comprehensive enough to be patched in but it's included below just in
        case it's helpful.

        Cheers,
        Brien

        Modification to the parseAllBlocks() function in IPTCParser.java.
        Modified area :

        int blockSize = bis
        .read4ByteInteger("Image Resource Block missing
        size");
        if (verbose)
        Debug.debug("blockSize", blockSize + " (0x"
        + Integer.toHexString(blockSize) + ")");
        //PATCH STARTS
        if(blockSize > bytes.length) // doesn't catch cases where
        blocksize is invalid but is still less than bytes.length but will at
        least prevent OutOfMemory errors
        throw new ImageReadException("Invalid Block Size :
        "+blockSize);
        //PATCH ENDS

        byte[] blockData = bis.readByteArray(blockSize,
        "Invalid Image Resource Block data", verbose, strict);

        Show
        Brien Voorhees added a comment - Thanks for looking into the issue. I actually did attach the image ('crash.jpeg') when I posted my comment. It appears to still be there (at the top of the page under "Image Attachments"). Just let me know if that doesn't work. I implemented a quick work around for the issue. Not sure if it's comprehensive enough to be patched in but it's included below just in case it's helpful. Cheers, Brien Modification to the parseAllBlocks() function in IPTCParser.java. Modified area : int blockSize = bis .read4ByteInteger("Image Resource Block missing size"); if (verbose) Debug.debug("blockSize", blockSize + " (0x" + Integer.toHexString(blockSize) + ")"); //PATCH STARTS if(blockSize > bytes.length) // doesn't catch cases where blocksize is invalid but is still less than bytes.length but will at least prevent OutOfMemory errors throw new ImageReadException("Invalid Block Size : "+blockSize); //PATCH ENDS byte[] blockData = bis.readByteArray(blockSize, "Invalid Image Resource Block data", verbose, strict);
        Hide
        Charles Matthew Chen added a comment -

        Brien Voorhees: can you please attach an image that demonstrates this issue?
        It should be fairly easy to resolve.

        Show
        Charles Matthew Chen added a comment - Brien Voorhees: can you please attach an image that demonstrates this issue? It should be fairly easy to resolve.
        Hide
        Charles Matthew Chen added a comment -

        I've committed a patch to address the issue Greg Squires's identified.

        http://svn.apache.org/viewcvs?view=rev&rev=995526

        Brien Voorhees' issue looks similar but unrelated, so I'm leaving this issue open.

        Show
        Charles Matthew Chen added a comment - I've committed a patch to address the issue Greg Squires's identified. http://svn.apache.org/viewcvs?view=rev&rev=995526 Brien Voorhees' issue looks similar but unrelated, so I'm leaving this issue open.
        Hide
        Brien Voorhees added a comment - - edited

        I'm trying to use Sanselan to analyze images in a live mailstream and unfortunately it crashes with OutOfMemory on malformed jpeg images. I'm using Sanselan version 0.97 on a 64-bit Mac.

        Attached is an example.

        Test code :
        Map params = new Hashtable();
        params.put(Sanselan.PARAM_KEY_VERBOSE, true);
        Sanselan.getMetadata(new File("crash.jpeg"), params);

        Below is the verbose output and a stack trace :

        blockType 1005 (0x3ed)
        blockSize 16 (0x10)
        blockType 1037 (0x40d)
        blockSize 4 (0x4)
        blockType 1011 (0x3f3)
        blockSize 8 (0x8)
        blockType 1034 (0x40a)
        blockSize 1 (0x1)
        blockType 10000 (0x2710)
        blockSize 10 (0xa)
        blockType 1013 (0x3f5)
        blockSize 72 (0x48)
        blockType 1016 (0x3f8)
        blockSize 112 (0x70)
        blockType 1032 (0x408)
        blockSize 16 (0x10)
        blockType 1044 (0x414)
        blockSize 4 (0x4)
        blockType 1036 (0x40c)
        blockSize 568 (0x238)
        blockType 4000 (0xfa0)
        blockSize 84 (0x54)
        blockType 4001 (0xfa1)
        blockSize 86622 (0x1525e)
        Invalid Image Resource Block data
        blockType 28519 (0x6f67)
        blockNameLength 108 (0x6c)
        blockSize 1811939328 (0x6c000000)

        Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
        at org.apache.sanselan.common.BinaryInputStream.readByteArray(BinaryInputStream.java:472)
        at org.apache.sanselan.formats.jpeg.iptc.IPTCParser.parseAllBlocks(IPTCParser.java:323)
        at org.apache.sanselan.formats.jpeg.iptc.IPTCParser.parsePhotoshopSegment(IPTCParser.java:116)
        at org.apache.sanselan.formats.jpeg.iptc.IPTCParser.parsePhotoshopSegment(IPTCParser.java:107)
        at org.apache.sanselan.formats.jpeg.segments.App13Segment.parsePhotoshopSegment(App13Segment.java:80)
        at org.apache.sanselan.formats.jpeg.JpegImageParser.getPhotoshopMetadata(JpegImageParser.java:607)
        at org.apache.sanselan.formats.jpeg.JpegImageParser.getMetadata(JpegImageParser.java:309)
        at org.apache.sanselan.Sanselan.getMetadata(Sanselan.java:978)
        at org.apache.sanselan.Sanselan.getMetadata(Sanselan.java:971)

        Show
        Brien Voorhees added a comment - - edited I'm trying to use Sanselan to analyze images in a live mailstream and unfortunately it crashes with OutOfMemory on malformed jpeg images. I'm using Sanselan version 0.97 on a 64-bit Mac. Attached is an example. Test code : Map params = new Hashtable(); params.put(Sanselan.PARAM_KEY_VERBOSE, true); Sanselan.getMetadata(new File("crash.jpeg"), params); Below is the verbose output and a stack trace : blockType 1005 (0x3ed) blockSize 16 (0x10) blockType 1037 (0x40d) blockSize 4 (0x4) blockType 1011 (0x3f3) blockSize 8 (0x8) blockType 1034 (0x40a) blockSize 1 (0x1) blockType 10000 (0x2710) blockSize 10 (0xa) blockType 1013 (0x3f5) blockSize 72 (0x48) blockType 1016 (0x3f8) blockSize 112 (0x70) blockType 1032 (0x408) blockSize 16 (0x10) blockType 1044 (0x414) blockSize 4 (0x4) blockType 1036 (0x40c) blockSize 568 (0x238) blockType 4000 (0xfa0) blockSize 84 (0x54) blockType 4001 (0xfa1) blockSize 86622 (0x1525e) Invalid Image Resource Block data blockType 28519 (0x6f67) blockNameLength 108 (0x6c) blockSize 1811939328 (0x6c000000) Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at org.apache.sanselan.common.BinaryInputStream.readByteArray(BinaryInputStream.java:472) at org.apache.sanselan.formats.jpeg.iptc.IPTCParser.parseAllBlocks(IPTCParser.java:323) at org.apache.sanselan.formats.jpeg.iptc.IPTCParser.parsePhotoshopSegment(IPTCParser.java:116) at org.apache.sanselan.formats.jpeg.iptc.IPTCParser.parsePhotoshopSegment(IPTCParser.java:107) at org.apache.sanselan.formats.jpeg.segments.App13Segment.parsePhotoshopSegment(App13Segment.java:80) at org.apache.sanselan.formats.jpeg.JpegImageParser.getPhotoshopMetadata(JpegImageParser.java:607) at org.apache.sanselan.formats.jpeg.JpegImageParser.getMetadata(JpegImageParser.java:309) at org.apache.sanselan.Sanselan.getMetadata(Sanselan.java:978) at org.apache.sanselan.Sanselan.getMetadata(Sanselan.java:971)

          People

          • Assignee:
            Unassigned
            Reporter:
            Greg Squires
          • Votes:
            1 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - 24h
              24h
              Remaining:
              Remaining Estimate - 24h
              24h
              Logged:
              Time Spent - Not Specified
              Not Specified

                Development