Uploaded image for project: 'Commons Compress'
  1. Commons Compress
  2. COMPRESS-466

Opening of a very large zip file is extremely slow compared to java.util.zip.ZipFile

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.18
    • Fix Version/s: 1.19
    • Component/s: Archivers
    • Labels:
    • Environment:

      Tested both on Linux and OSX 10.13.6.

      Description

      We have a quite large zip file 35 gb and try to open this with ZipFile. 

              try (ZipFile zf = new ZipFile(new File("35gb.zip"))) {
                  System.out.println("File opened..." + (System.currentTimeMillis() - start));
              }
      

      This code takes about 300 000 - 400 000 ms (5-6 minutes).
      If I run this with JDK-builtin java.util.zip.ZipFile, same code takes 300 ms (less than a second).

      I'm not totally sure what it is the problem but I did some debugging and basically all time is spent in

          private void resolveLocalFileHeaderData(final Map<ZipArchiveEntry, NameAndComment> entriesWithoutUTF8Flag)
      

      Anything that can be done to improve this?

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              jakeri Jakob Sultan Ericsson
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: