[HADOOP-6662] hadoop zlib compression does not fully utilize the buffer

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 0.20.2
• Component/s: io
• Labels: compress hadoop io zlib
• Tags: hadoop io compress zlib
• Assignee: Unassigned
• Reporter: Kang Xiao
• Created: 29/Mar/10 07:58
• Updated: 06/Apr/10 02:12
• Resolved: 06/Apr/10 02:12

Description

org.apache.hadoop.io.compress.ZlibCompressonr does not fully utilize its buffer.

Its needesInput() return false when there is any data in its buffer (64K by default). The performance will greately degrade since an JNI call will be invoded at each time the write() method of CompressonStream is called.