[IMPALA-2807] During insert operation impala creates too many files for a table size < block size

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: Impala 2.3.0
• Component/s: Perf Investigation
• Target Version: Product Backlog
• Assignee: Unassigned
• Reporter: Dileep Kumar
• Created: 04/Jan/16 23:21
• Updated: 21/Dec/20 18:05
• Resolved: 21/Dec/20 18:05

Description

When loading the "customer" table from TPC-DS based schema, total no. of files created is 20 (which is equal to number of impala nodes in the cluster).
The total size of the this table is 204.2 MiB which can fit in a single block while it occupies 20 blocks in this case.
When ran the same insert command with a single impalad running in the cluster single block was able to hold all the table data and only one hdfs file was created.