[HIVE-19457] Support Iceberg as a table format

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Component/s: Metastore, StorageHandler
• Release Note: Support added for the iceberg table format.
• Assignee: Unassigned
• Reporter: Elliot West
• Created: 08/May/18 09:10
• Updated: 07/Dec/22 22:00
• Resolved: 07/Dec/22 22:00

Description

From Netflix/iceberg:

Iceberg is a new table format for storing large, slow-moving tabular data. It is designed to improve on the de-facto standard table layout built into Hive, Presto, and Spark.

Iceberg elegantly solves many data and metadata challenges when handling large datasets, especially when using eventually consistent file stores. By integrating iceberg with Hive we could hope to gain the following:

• Improved performance when reading large numbers of partitions (HMS is no longer a bottleneck).
• Users receiving a consistent view when reading changing datasets (snapshot isolation).
• Strong consistency on eventually consistent file stores without the need for additional infrastructure (data is immutable, eventually consistent ops are avoided; data is always PUT and LIST is avoided).

Issue Links

• is superceded by: HIVE-24867 Create iceberg-handler module in Hive