[IMPALA-6200] Flakiness in Planner Tests

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: Impala 2.11.0
• Component/s: Frontend
• Labels: broken-build flaky
• Target Version: Impala 2.11.0
• Epic Color: ghx-label-2
• Assignee: Zach Amsden
• Reporter: Taras Bobrovytsky
• Created: 16/Nov/17 03:43
• Updated: 21/Nov/17 22:45
• Resolved: 21/Nov/17 22:45

Description

Sometimes we are seeing random small variations in Planner tests, which causes builds to be flaky.

Actual:

F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
|  Per-Host Resources: mem-estimate=48.00MB mem-reservation=0B
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
WRITE TO HDFS [functional.alltypes, OVERWRITE=false, PARTITION-KEYS=(CAST(3 + year AS INT),CAST(month - -1 AS INT))]
|  partitions=4
|  mem-estimate=1.56KB mem-reservation=0B

Expected:

F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
|  Per-Host Resources: mem-estimate=32.00MB mem-reservation=0B
WRITE TO HDFS [functional.alltypes, OVERWRITE=false, PARTITION-KEYS=(CAST(3 + year AS INT),CAST(month - -1 AS INT))]
|  partitions=4
|  mem-estimate=1.56KB mem-reservation=0B

Actual:

|  F01:PLAN FRAGMENT [RANDOM] hosts=2 instances=2
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|  Per-Host Resources: mem-estimate=16.00MB mem-reservation=0B
|  01:SCAN HDFS [functional_parquet.alltypestiny, RANDOM]
|     partitions=4/4 files=4 size=9.75KB
|     stats-rows=unavailable extrapolated-rows=disabled
|     table stats: rows=unavailable size=unavailable
|     column stats: unavailable
|     mem-estimate=16.00MB mem-reservation=0B
|     tuple-ids=1 row-size=88B cardinality=unavailable

Expected:

|  F01:PLAN FRAGMENT [RANDOM] hosts=3 instances=3
|  Per-Host Resources: mem-estimate=16.00MB mem-reservation=0B
|  01:SCAN HDFS [functional_parquet.alltypestiny, RANDOM]
|     partitions=4/4 files=4 size=10.48KB
|     stats-rows=unavailable extrapolated-rows=disabled
|     table stats: rows=unavailable size=unavailable
|     column stats: unavailable
|     mem-estimate=16.00MB mem-reservation=0B
|     tuple-ids=1 row-size=88B cardinality=unavailable

Issue Links

• duplicates: IMPALA-3887 Planner tests failing due to metadata loading race with HDFS, fewer #hosts than expected