[HIVE-27005] Iceberg: Col stats are not used in queries

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Iceberg integration
• Labels: iceberg performance
• Assignee: Simhadri Govindappa
• Reporter: Rajesh Balamohan
• Created: 31/Jan/23 06:42
• Updated: 31/Jan/23 10:12

Description

1. Though, insert-queries compute colstats during runtime, they are not persisted in HMS during final call. 

2. Due to #1, col stats are not available during runtime for hive queries. This includes col stats, NDV etc. So unless users explicitly run "analyse table" statements, queries can be have suboptimal plans.

E.g col_stats.txt(note that there is no col stats being used)