[COMPRESS-162] BZip2CompressorInputStream still stops after 900,000 decompressed bytes of large compressed file

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 1.3
• Fix Version/s: 1.4
• Component/s: Compressors

Environment

Linux (Fedora Cores 13 [2.6.34.9-69.fc13.i686.PAE] and 15, at latest 'yum upgrade' as of 7 Nov 2011), Sun Java 1.6.0_22

• Assignee: Unassigned
• Reporter: Andrew Pavlin
• Created: 08/Nov/11 05:13
• Updated: 09/Nov/11 15:53
• Resolved: 09/Nov/11 15:53

Description

Attempting to unzip the planet-110921.osm.bz2 file downloaded directly from planet.OpenStreetMaps.org aborts after exactly 900000 bytes are uncompressed. The uncompressed content looks like valid XML, and causes my application's parser to blow up with XML syntax errors due to missing closing tags. Tried using the example code to just uncompress, and got the same exact behavior.

Uncompressing the same file planet-110921.osm.bz2 (19357793489 bytes long compressed) with the Linux bzip2 command-line utility (bzip2-1.0.6-1.fc13.i686.rpm) succeeds and produces a valid (and enormous) XML file that can be successfully parsed.

Tried getting a subversion snapshot of the commons-compress trunk on 7 Nov 2011 and replacing the org.apache.commons.compress.compressors.bzip2 package in the commons-compress-1.3.jar with compiled code from the trunk (Subversion log reported that the fix for COMPRESS-146 was in). Still the same failure.

Issue Links

• duplicates: COMPRESS-146 BZip2CompressorInputStream always treats 0x177245385090 as EOF, but should treat this as EOS