[DRILL-6839] Failed to plan (aggregate + Hash or NL join) when slice target is low

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Igor Guzenko
• Reporter: Igor Guzenko
• Created: 09/Nov/18 12:34
• Updated: 04/Nov/19 12:48

Description

Case 1. When nested loop join is about to be used:

• Option "planner.enable_nljoin_for_scalar_only" is set to false
• Option "planner.slice_target" is set to low value for imitation of big input tables

 

@Category(SqlTest.class)
public class CrossJoinTest extends ClusterTest {
 @BeforeClass
 public static void setUp() throws Exception {
 startCluster(ClusterFixture.builder(dirTestWatcher));
 }

 @Test
 public void testCrossJoinSucceedsForLowSliceTarget() throws Exception {
   try {
     client.alterSession(PlannerSettings.NLJOIN_FOR_SCALAR.getOptionName(), false);
     client.alterSession(ExecConstants.SLICE_TARGET, 1);
     queryBuilder().sql(
        "SELECT COUNT(l.nation_id) " +
        "FROM cp.`tpch/nation.parquet` l " +
        ", cp.`tpch/region.parquet` r")
     .run();
   } finally {
    client.resetSession(ExecConstants.SLICE_TARGET);
    client.resetSession(PlannerSettings.NLJOIN_FOR_SCALAR.getOptionName());
   }
 }
}

 

Case 2. When hash join is about to be used:

• Option "planner.enable_mergejoin" is set to false, so hash join will be used instead
• Option "planner.slice_target" is set to low value for imitation of big input tables
• Comment out //ruleList.add(HashJoinPrule.DIST_INSTANCE); in PlannerPhase.getPhysicalRules method

  @Category(SqlTest.class)
  public class CrossJoinTest extends ClusterTest {
   @BeforeClass
   public static void setUp() throws Exception {
     startCluster(ClusterFixture.builder(dirTestWatcher));
   }
  
   @Test
   public void testInnerJoinSucceedsForLowSliceTarget() throws Exception {
     try {
      client.alterSession(PlannerSettings.MERGEJOIN.getOptionName(), false);
      client.alterSession(ExecConstants.SLICE_TARGET, 1);
      queryBuilder().sql(
        "SELECT COUNT(l.nation_id) " +
        "FROM cp.`tpch/nation.parquet` l " +
        "INNER JOIN cp.`tpch/region.parquet` r " +
        "ON r.nation_id = l.nation_id")
      .run();
     } finally {
      client.resetSession(ExecConstants.SLICE_TARGET);
      client.resetSession(PlannerSettings.MERGEJOIN.getOptionName());
     }
   }
  }
  

   

Workaround: To avoid the exception we need to set option "planner.enable_multiphase_agg" to false. By doing this we avoid unsuccessful attempts to create 2 phase aggregation plan in StreamAggPrule and guarantee that logical aggregate will be converted to physical one.