[IMPALA-12408] Planner spends lot of time in HdfsPartition.getLocation()

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 4.3.0
• Component/s: Frontend
• Labels: performance
• Epic Color: ghx-label-6
• Assignee: Csaba Ringhofer
• Reporter: Csaba Ringhofer
• Created: 28/Aug/23 16:58
• Updated: 15/Sep/23 17:57
• Resolved: 15/Sep/23 17:57

Description

For queries with lot of partitions the majority of planning time can be spent decoding compressed partition locations. This can be often avoided, e.g. by caching FsType instead of always decompressing the path to get its prefix: https://github.com/apache/impala/blob/218c4c447eadb14fadb8310db4b46ab8c04cb1ba/fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java#L914