[COMPRESS-592] Checksum verification failed reading 7z archive with more than 65536 entries

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.21
• Fix Version/s: 1.22
• Component/s: Compressors

Environment

Compress 1.21 and XZ 1.9 on JDK 11; reproduced on both Windows and Ubuntu Linux

Language

Java

• Assignee: Unassigned
• Reporter: Roland Kreuzer
• Created: 27/Sep/21 13:31
• Updated: 30/Oct/21 01:03
• Resolved: 30/Oct/21 01:03

Description

I have a use-case where I have to decompress Sevenzip archives from an external source which may have a large number of entries.

I found decompression fails when trying to extract entry 65536 (zero-based index) with a checksum failure.

 

I was able to reproduce the issue with a simple 7Zip file containing 70.001 entries with random MD5 checksum textfiles (attached).

The sample Archive was created using the 7Zip Windows client and uses LZMA2:3m.

 

My code is a simple sequential read of all contents of the file like

    @Test
    void readBigSevenZipFile() throws IOException
    {
        try (SevenZFile sevenZFile = new SevenZFile(new File("E:\\Temp\\0000_DOC.7z")))
        {
            SevenZArchiveEntry entry = sevenZFile.getNextEntry();
            while (entry != null)
            {
                if (entry.hasStream())
                {
                    byte[] content = new byte[(int) entry.getSize()];
                    sevenZFile.read(content);

                    System.out.println(entry.getName());
                }

                entry = sevenZFile.getNextEntry();
            }
        }
    }

which fails consistently after file65535.txt with

java.io.IOException: Checksum verification failed
        at org.apache.commons.compress.utils.ChecksumVerifyingInputStream.read(ChecksumVerifyingInputStream.java:94) ~[commons-compress-1.21.jar!/:1.21]
        at org.apache.commons.compress.archivers.sevenz.SevenZFile.read(SevenZFile.java:1905) ~[commons-compress-1.21.jar!/:1.21]
        at org.apache.commons.compress.archivers.sevenz.SevenZFile.read(SevenZFile.java:1888) ~[commons-compress-1.21.jar!/:1.21]

 

It is noticeable that the value is 2 to the 16th power, which could suggest an overflow error of some sorts.

 

While the minimal sample contains only small txt files, I originally found the issue with larger archives containing also Image and PDF files. The archive's contents or size in byte does not seem to have direct influence on the issue, only the number of files contained within.

 

I did not find any workaround yet.