Description
This is a followup for HIVE-7053. Currently the code to split the operator tree and to generate tasks is mingled and thus hard to understand and maintain. Logically the two seems independent. This can be improved by modulizing both. The following might be helpful:
@Override protected void generateTaskTree(List<Task<? extends Serializable>> rootTasks, ParseContext pCtx, List<Task<MoveWork>> mvTask, Set<ReadEntity> inputs, Set<WriteEntity> outputs) throws SemanticException { // 1. Identify if the plan is for multi-insert and split the plan if necessary List<Set<Operator>> operatorSets = multiInsertSplit(...); // 2. For each operator set, generate a task. for (Set<Operator> topOps : operatorSets) { SparkTask task = generateTask(topOps); ... } // 3. wire up the tasks ... }
No needed with
HIVE-8118.