[HIVE-5783] Native Parquet Support in Hive

• Type: New Feature
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 0.13.0
• Component/s: Serializers/Deserializers
• Labels: Parquet
• Release Note: Added support for 'STORED AS PARQUET' and for setting parquet as the default storage engine.
• Assignee: Justin Coffey Assign to me
• Reporter: Justin Coffey
• Created: 08/Nov/13 18:40
• Updated: 29/Jun/21 10:14
• Resolved: 09/Feb/14 00:26

Description

Problem Statement:

Hive would be easier to use if it had native Parquet support. Our organization, Criteo, uses Hive extensively. Therefore we built the Parquet Hive integration and would like to now contribute that integration to Hive.

About Parquet:

Parquet is a columnar storage format for Hadoop and integrates with many Hadoop ecosystem tools such as Thrift, Avro, Hadoop MapReduce, Cascading, Pig, Drill, Crunch, and Hive. Pig, Crunch, and Drill all contain native Parquet integration.

Changes Details:

Parquet was built with dependency management in mind and therefore only a single Parquet jar will be added as a dependency.

Issue Links

• blocks: HIVE-6367 Implement Decimal in ParquetSerde
• blocks: HIVE-6375 Fix CTAS for parquet
• blocks: HIVE-6366 Refactor some items in Hive Parquet
• blocks: HIVE-6384 Implement all Hive data types in Parquet
• is related to: HIVE-25296 Replace parquet-hadoop-bundle dependency with the actual parquet modules
• is related to: HIVE-5976 Decouple input formats from STORED as keywords
• relates to: HIVE-5998 Add vectorized reader for Parquet files
• relates to: HIVE-6368 Document parquet on hive wiki