[HIVE-5504] OrcOutputFormat honors compression properties only from within hive

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.11.0, 0.12.0, 0.13.0
• Fix Version/s: 0.13.0
• Component/s: HCatalog

Release Note

Allows OrcOutputFormat.createRecordWriter() to use its table property parameters (like "orc.compress") if specified in JobConf as well. This codepath is not called from Hive, but will be helpful to those that want to use OrcOutputFormat as a generic M/R OutputFormat.

In addition, makes changes to HCatalog to look for certain Orc table properties, and if present, copies them out to JobConf before instantiating OrcOutputFormat, thus passing the properties back to it. This allows users of Orc from outside Hive to have Orc behave as the metadata dictates.

• Assignee: Sushanth Sowmyan
• Reporter: Venkat Ranganathan
• Created: 09/Oct/13 18:46
• Updated: 03/Mar/14 18:55
• Resolved: 03/Mar/14 18:55

Description

When we import data into a HCatalog table created with the following storage description

.. stored as orc tblproperties ("orc.compress"="SNAPPY")
the resultant orc file still uses the default zlib compression

It looks like HCatOutputFormat is ignoring the tblproperties specified. show tblproperties shows that the table indeed has the properties properly saved.

An insert/select into the table has the resulting orc file honor the tbl property.

Issue Links

• relates to: HIVE-6506 hcatalog should automatically work with new tableproperties in ORC
• relates to: HIVE-6507 OrcFile table property names are specified as strings