[HIVE-28596] Skip ColumnAccessInfo collection when not needed to speed-up compilation

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.1.0
• Security Level: Public (Viewable by anyone)
• Labels: pull-request-available
• Assignee: Stamatis Zampetakis
• Reporter: Stamatis Zampetakis
• Created: 29/Oct/24 14:40
• Updated: 04/Nov/24 10:20
• Resolved: 04/Nov/24 10:18

Description

Avoid paying the perf-overhead of collecting ColumnAccessInfo via the HiveRelFieldTrimmer when it is not necessary.

Currently, we only use the ColumnAccessInfo when we need to perform column based authorization or when it is requested explicitly by the user via the hive.stats.collect.scancols property.

The field trimmer is an expensive operation and can become pretty slow for certain use-cases/queries so skipping some calls when it is not necessary is in general beneficial and can notably improve the compilation time for some queries.

Issue Links

• relates to: HIVE-28582 OOM when compiling query with many GROUP BY columns aliased multiple times
• links to: GitHub Pull Request #5517