[IMPALA-5311] Select count(*) queries show in incorrect compression in profile

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: Impala 2.9.0
• Fix Version/s: Impala 2.11.0
• Component/s: Backend
• Labels: ramp-up supportability
• Epic Color: ghx-label-4
• Assignee: Anuj Phadke
• Reporter: Peter Ebert
• Created: 11/May/17 21:51
• Updated: 21/Sep/17 17:57
• Resolved: 21/Sep/17 17:57

Description

If you run a count query you will see the following in the profile:
File Formats: PARQUET/NONE:1

if you then run a select * you will see the correct compression format:
File Formats: PARQUET/SNAPPY:14

If the correct compression cannot be inferred without reading the data (presuming this happens because Impala just reads the parquet stats) might be better to put nothing at all instead of none. Ideally though SNAPPY would display even on count.

Issue Links

• is related to: IMPALA-4863 Incorrect accounting of file count and compression type when Runtime filters are applied on partition and non-partition columns
• relates to: IMPALA-4863 Incorrect accounting of file count and compression type when Runtime filters are applied on partition and non-partition columns