[HIVE-85] separate compression options for different output types

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.3.0
• Component/s: Query Processor
• Labels: compression
• Assignee: Joydeep Sen Sarma
• Reporter: Joydeep Sen Sarma
• Created: 27/Nov/08 18:59
• Updated: 17/Dec/11 00:08
• Resolved: 05/Dec/08 09:37

Description

currently hive uses mapred.output.compress to determine compression for all output files. however not all files are final output. at least three different kinds of output files are generated:
1. intermediate output files for next map-reduce job
2. files targeted for result hdfs directories or hive tables/partitions (which are just hdfs dirs)
3. files written to user local directories (downloading results)

the plan is to provide three separate options for controlling 1,2,3 separately. we may want to split (2) in case compression is determined by table metadata (and not session options).