[HUDI-1297] [Umbrella] Spark Datasource Support

• Type: Epic
• Status: Open
• Priority: Blocker
• Resolution: Unresolved
• Affects Version/s: 0.9.0
• Component/s: spark
• Labels: hudi-umbrellas
• Epic Name: Hudi Spark Datasource
• Assignee: Vinoth Chandar
• Reporter: Vinoth Chandar
• Created: 23/Sep/20 02:45
• Updated: 03/Oct/23 03:35

Description

Yet to be fully scoped out

But high level, we want to 

• First class support for streaming reads/writes via structured streaming
• Row based reader/writers all the way
• Support for File/Partition pruning using Hudi metadata tables

Issue Links

• is a parent of: HUDI-1040 Support Spark3 compatibility
• is blocked by: HUDI-1292 [Umbrella] RFC-15 : Metadata Table for File Listing and other table metadata