[IMPALA-8021] Add estimated cardinality to EXPLAIN output

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: Impala 3.1.0
• Fix Version/s: Impala 3.2.0
• Component/s: Frontend
• Epic Color: ghx-label-2
• Assignee: Paul Rogers
• Reporter: Paul Rogers
• Created: 27/Dec/18 22:20
• Updated: 17/Oct/19 13:39
• Resolved: 07/Feb/19 01:10

Description

The EXPLAIN output provides much useful information in the plan tree. All our planning decisions are based on cardinality; but it appears in the EXPLAIN output only for the EXTENDED level. The profile only contains the plan from the STANDARD level. This change proposes to include row size and cardinality even in the STANDARD level.

The nodes that have the information call it "cardinality", so continue to use that term.

Add cardinality to each node so it appears something like this:

HASH JOIN [INNER JOIN, BROADCAST]
|  row-size=89B cardinality=1.23G
|
|--SCAN HDFS [db.table]
|     partitions=2/123 files=2 size=4.56MB row-size=89B cardinality=7.89M

Cardinality should appear in all levels above MINIMAL. Cardinality is not needed for EXCHANGE since it can be inferred from other nodes.

Also, the existing code prints large cardinalities in detail: 1234567890, which is hard to read. Use the abbreviated output, using metric (power of 1000) units, so 1.23G instead.

Issue Links

• breaks: IMPALA-8986 Inconsistent metric printing between exec summary and EXPLAIN output
• breaks: IMPALA-8078 test_corrupt_stats failing on exhaustive builds
• incorporates: IMPALA-7919 Add predicates line in plan output for partition key predicates
• incorporates: IMPALA-8022 Add cardinality checks to PlannerTest
• incorporates: IMPALA-8023 Fix PlannerTest to handle error lines consistently