Commons Compress
  1. Commons Compress
  2. COMPRESS-162

BZip2CompressorInputStream still stops after 900,000 decompressed bytes of large compressed file

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Duplicate
    • Affects Version/s: 1.3
    • Fix Version/s: 1.4
    • Component/s: Compressors
    • Labels:
      None
    • Environment:

      Linux (Fedora Cores 13 [2.6.34.9-69.fc13.i686.PAE] and 15, at latest 'yum upgrade' as of 7 Nov 2011), Sun Java 1.6.0_22

      Description

      Attempting to unzip the planet-110921.osm.bz2 file downloaded directly from planet.OpenStreetMaps.org aborts after exactly 900000 bytes are uncompressed. The uncompressed content looks like valid XML, and causes my application's parser to blow up with XML syntax errors due to missing closing tags. Tried using the example code to just uncompress, and got the same exact behavior.

      Uncompressing the same file planet-110921.osm.bz2 (19357793489 bytes long compressed) with the Linux bzip2 command-line utility (bzip2-1.0.6-1.fc13.i686.rpm) succeeds and produces a valid (and enormous) XML file that can be successfully parsed.

      Tried getting a subversion snapshot of the commons-compress trunk on 7 Nov 2011 and replacing the org.apache.commons.compress.compressors.bzip2 package in the commons-compress-1.3.jar with compiled code from the trunk (Subversion log reported that the fix for COMPRESS-146 was in). Still the same failure.

        Issue Links

          Activity

          Hide
          Stefan Bodewig added a comment -

          Andrew, are you using the two-arg constructor for BZip2CompressorInputStream? Concatenated files are not supported by default but only when you ask for it.

          Show
          Stefan Bodewig added a comment - Andrew, are you using the two-arg constructor for BZip2CompressorInputStream? Concatenated files are not supported by default but only when you ask for it.
          Hide
          Andrew Pavlin added a comment -

          switched to the two-arg constructor with true for the second argument, and now it isn't stopping (yet). I'll provide further feedback after it finishes the file (my app takes about 12 hours to parse the entire planet.osm file).

          Show
          Andrew Pavlin added a comment - switched to the two-arg constructor with true for the second argument, and now it isn't stopping (yet). I'll provide further feedback after it finishes the file (my app takes about 12 hours to parse the entire planet.osm file).
          Hide
          Andrew Pavlin added a comment -

          Turns out everything works just fine when I use the 2-arg constructor. Please close this issue as a duplicate of 146.

          Show
          Andrew Pavlin added a comment - Turns out everything works just fine when I use the 2-arg constructor. Please close this issue as a duplicate of 146.
          Hide
          Stefan Bodewig added a comment -

          Thank you for checking.

          Show
          Stefan Bodewig added a comment - Thank you for checking.

            People

            • Assignee:
              Unassigned
              Reporter:
              Andrew Pavlin
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development