[PIG-4600] Delegate bzip processing to Hadoop

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Fix Version/s: 0.16.0
• Assignee: Daniel Dai
• Reporter: Daniel Dai
• Created: 12/Jun/15 18:12
• Updated: 08/Jun/16 20:48
• Resolved: 12/Jun/15 18:21

Description

Pig handling bzip within its own codebase. I am not very sure but that's probably due to history reason. Hadoop does not support splittable bzip until 0.21 (HADOOP-4012). If so, we shall remove the bzip2 module in Pig and delegate to Hadoop.

In particular, Pig does not support bzip2 concatenation. Once we delegate bzip to Hadoop, this issue will be solved.

Issue Links

• duplicates: PIG-4591 Drop use of the internal Bzip2TextInputFormat