[IMPALA-7442] test_semi_joins_exhaustive occasionally fails

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 3.1.0
• Fix Version/s: Impala 3.1.0
• Component/s: Backend
• Labels: flakey
• Target Version: Impala 3.1.0
• Epic Color: ghx-label-1
• Assignee: Tim Armstrong
• Reporter: Michael Brown
• Created: 14/Aug/18 19:01
• Updated: 16/Aug/18 00:30
• Resolved: 16/Aug/18 00:29

Description

This test occasionally fails with a very large error message, the top of which is shared below. I will attached a gzip profile of my local reproduction. I've seen this test fail twice on a release build of Impala. The reproduction and profile I shared below was also with a release build.

Cannot perform hash join at node with id 4. Repartitioning did not reduce the size of a spilled partition. Repartitioning level 1. Number of rows 60280000:
PartitionedHashJoinNode (id=4 op=2 state=RepartitioningBuild #spilled_partitions=0)
PhjBuilder: Hash partitions: 16:

I was able to reproduce this by doing:

for f in $(seq 100)
do
  impala-py.test -k test_semi_joins_exhaustive tests/query_test/test_join_queries.py --workload_exploration_strategy functional-query:exhaustive -m "not execute_serially" || { echo $f; break; }
done

If you want more noise but a faster reproduction, try:

impala-py.test -k test_semi_joins_exhaustive tests/query_test/test_join_queries.py --workload_exploration_strategy functional-query:exhaustive -m "not execute_serially" -n4

Issue Links

• relates to: IMPALA-7446 Queries can spill earlier than necessary because of accumulation of free buffers and clean pages