[IMPALA-3742] INSERTs into Kudu tables should partition and sort

• Type: Improvement
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Kudu_Impala
• Fix Version/s: Impala 2.9.0
• Component/s: Backend
• Labels: kudu performance
• Target Version: Impala 2.9.0
• Assignee: Thomas Tauber-Marshall
• Reporter: Matthew Jacobs
• Created: 14/Jun/16 20:31
• Updated: 01/Sep/17 21:45
• Resolved: 04/May/17 16:00

Description

Inserts into Kudu tables should be partitioned (i.e. rows hashed using the same hash partitioning as the Kudu table) and, at the table sink, sorted on the primary key. This would significantly improve performance.

This will require a local sort (IMPALA-2521), and support from Kudu to provide the partitioning.

Issue Links

• breaks: IMPALA-5871 KuduPartitionExpr incorrectly handles its child types
• breaks: IMPALA-5294 Kudu INSERT partitioning fails with constants
• breaks: IMPALA-5611 KuduPartitionExpr holds onto memory unnecessarily
• is related to: IMPALA-2521 Introduce CLUSTERED plan hint for insert statements