[HUDI-8679] Enable partition stats and col stats by default - Phase 0

• Type: Improvement
• Status: In Progress
• Priority: Blocker
• Resolution: Unresolved
• Fix Version/s: 1.0.1
• Component/s: metadata
• Labels: pull-request-available story
• Assignee: sivabalan narayanan
• Reporter: sivabalan narayanan
• Created: 09/Dec/24 17:55
• Updated: 17/Dec/24 04:41

Description

Lets enable col stats by default on the writer

Issue Links

• links to: GitHub Pull Request #12453
• links to: GitHub Pull Request #12465
• links to: GitHub Pull Request #12472
• links to: GitHub Pull Request #12479

Sub-Tasks

• Add table property to track list of columns indexed by partition stats: Open
• Enable partition stats by default on writer: Patch Available
• Do col stats lookup only for indexed cols: Open
• ColumnStats decimal record type not supported: Open
• Data type promotion does not work w/ partition stats index: Open
• Fix unsupported data types for SPARK record type w/ col stats: Open
• Unify the unsupported types across SPARK and AVRO record type: Patch Available
• Add nested fields w/ col stats across base and log files: Open
• Fix Partition stats for "Date" datatype: Open
• Optimize re-adding missing files to col stats pruning: Open
• Apply clean commits to col stats partition on restore in data table: Open
• Add table property to track list of columns being indexed in col stats: Patch Available
• Enable col stats by default on the writer: Patch Available
• Add more data type support w/ col stats: Patch Available