[HUDI-5364] Make sure Hudi's Column Stats are wired into Spark's relation stats

• Type: Bug
• Status: Open
• Priority: Critical
• Resolution: Unresolved
• Affects Version/s: 0.12.1
• Fix Version/s: 1.1.0
• Component/s: spark, spark-sql
• Story Points: 4
• Assignee: Alexey Kudinkin
• Reporter: Alexey Kudinkin
• Created: 10/Dec/22 02:11
• Updated: 21/Dec/23 15:28

Description

Currently, we're leveraging CSI exclusively to better prune the target files.

Additionally, we should wire in stats from CSI into Spark's `CatalogStatistics` which in turn will be leveraged by Spark's Optimization rules for better planning.