[HUDI-8724] Bug fixes - Phase 1

• Type: Improvement
• Status: Open
• Priority: Critical
• Resolution: Unresolved
• Fix Version/s: 1.0.1
• Labels: story
• Assignee: Unassigned
• Reporter: Y Ethan Guo
• Created: 12/Dec/24 00:23
• Updated: 13/Dec/24 01:56

Sub-Tasks

• Difference between HoodieData.map and mapPartitions: Open
• Fix ORC tests on Spark 3.5: Open
• Fix JSON Reader tests for Spark > 3.3: Open
• Incorrect partition pruning when TimestampBasedKeyGenerator is used in partition column: Open
• Incorrect type casting while reading HUDI table created with CustomKeyGenerator and unixtimestamp paritioning field: Open
• Support YYYY-MM-DD partition format with hive: Open
• Support YYYY/MM/DD partition format with hive: Open
• Different parquet reader config on list-typed fields is used to read parquet file generated by clustering: Open
• Writer failing with exception when precision is different for two decimal fields with same name: Open
• Using CustomKeyGenerator fails w/ SparkHoodieTableFileIndex: In Progress
• Use snapshot query as default in DefaultSource: Patch Available
• Fix log file marker creation in file group initialization in metadata table writer: In Progress
• Fix default value of bootstrap table config: In Progress
• Revisit commitsMetadata fetching from timeline history in MergeOnReadIncrementalRelation: Open
• Reduce spurious logs from Spark SQL write: Open
• Proper cleanup of BitCaskDiskMap on storage: Open
• Follow up with EMR to fix out-of-the-box experience for Hudi streamer continuous mode: Open