[HUDI-2832] [Umbrella] [RFC-40] Integrated Hudi with Snowflake

• Type: Epic
• Status: In Progress
• Priority: Critical
• Resolution: Unresolved
• Component/s: Common Core
• Labels: BigQuery Integration pull-request-available
• Story Points: 0
• Epic Name: [RFC-40] Implement SnowflakeSyncTool to support Hudi to Snowflake Integration
• Assignee: Vinoth Govindarajan
• Reporter: Vinoth Govindarajan
• Created: 22/Nov/21 20:24
• Updated: 21/Dec/23 15:33

Description

Snowflake is a fully managed service that’s simple to use but can power a near-unlimited number of concurrent workloads. Snowflake is a solution for data warehousing, data lakes, data engineering, data science, data application development, and securely sharing and consuming shared data. Snowflake doesn’t support Apache Hudi file format yet, but it has support for Parquet, ORC, and Delta file format. This proposal is to implement a SnowflakeSync similar to HiveSync to sync the Hudi table as the Snowflake External Parquet table so that users can query the Hudi tables using Snowflake. Many users have expressed interest in Hudi and other support channels asking to integrate Hudi with Snowflake, this will unlock new use cases for Hudi.

Issue Links

• links to: GitHub Pull Request #4074