Hive
  1. Hive
  2. HIVE-4827

Merge a Map-only task to its child task

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.12.0
    • Fix Version/s: 0.12.0
    • Component/s: Query Processor
    • Labels:
      None
    • Release Note:
      Hide
      Before applying this jira to trunk, CommonJoinTaskDispatcher has two methods, mergeMapJoinTaskWithChildMapJoinTask and mergeMapJoinTaskWithMapReduceTask. The first method tries to merge a map-only task (for MapJoin) to its child map-only task. The second method tries to merge a map-only task to its child MapReduce task (a task has a reducer). There was a flag called "hive.optimize.mapjoin.mapreduce" to determine if mergeMapJoinTaskWithMapReduceTask will be called.

      This work combines mergeMapJoinTaskWithChildMapJoinTask and mergeMapJoinTaskWithMapReduceTask. So, a map-only task will be merged into its child task no matter the child task is a map-only task or a MapReduce task. So hive.optimize.mapjoin.mapreduce is not needed any more.

      If a user wants to disable merging a map-only task to its child task, he or she can use either
      set hive.auto.convert.join.noconditionaltask=false;
      or
      set hive.auto.convert.join.noconditionaltask=true;
      set hive.auto.convert.join.noconditionaltask.size=0;
      Show
      Before applying this jira to trunk, CommonJoinTaskDispatcher has two methods, mergeMapJoinTaskWithChildMapJoinTask and mergeMapJoinTaskWithMapReduceTask. The first method tries to merge a map-only task (for MapJoin) to its child map-only task. The second method tries to merge a map-only task to its child MapReduce task (a task has a reducer). There was a flag called "hive.optimize.mapjoin.mapreduce" to determine if mergeMapJoinTaskWithMapReduceTask will be called. This work combines mergeMapJoinTaskWithChildMapJoinTask and mergeMapJoinTaskWithMapReduceTask. So, a map-only task will be merged into its child task no matter the child task is a map-only task or a MapReduce task. So hive.optimize.mapjoin.mapreduce is not needed any more. If a user wants to disable merging a map-only task to its child task, he or she can use either set hive.auto.convert.join.noconditionaltask=false; or set hive.auto.convert.join.noconditionaltask=true; set hive.auto.convert.join.noconditionaltask.size=0;

      Description

      When hive.optimize.mapjoin.mapreduce is on, CommonJoinResolver can attach a Map-only job (MapJoin) to its following MapReduce job. But this merge only happens when the MapReduce job has a single input. With Correlation Optimizer (HIVE-2206), it is possible that the MapReduce job can have multiple inputs (for multiple operation paths). It is desired to improve CommonJoinResolver to merge a Map-only job to the corresponding Map task of the MapReduce job.

      Example:

      set hive.optimize.correlation=true;
      set hive.auto.convert.join=true;
      set hive.optimize.mapjoin.mapreduce=true;
      SELECT tmp1.key, count(*)
      FROM (SELECT x1.key1 AS key
            FROM bigTable1 x1 JOIN smallTable1 y1 ON (x1.key1 = y1.key1)
            GROUP BY x1.key1) tmp1
      JOIN (SELECT x2.key2 AS key
            FROM bigTable2 x2 JOIN smallTable2 y2 ON (x2.key2 = y2.key2)
            GROUP BY x2.key2) tmp2
      ON (tmp1.key = tmp2.key)
      GROUP BY tmp1.key;
      

      In this query, join operations inside tmp1 and tmp2 will be converted to two MapJoins. With Correlation Optimizer, aggregations in tmp1, tmp2, and join of tmp1 and tmp2, and the last aggregation will be executed in the same MapReduce job (Reduce side). Since this MapReduce job has two inputs, right now, CommonJoinResolver cannot attach two MapJoins to the Map side of a MapReduce job.

      Another example:

      SELECT tmp1.key
      FROM (SELECT x1.key2 AS key
            FROM bigTable1 x1 JOIN smallTable1 y1 ON (x1.key1 = y1.key1)
            UNION ALL
            SELECT x2.key2 AS key
            FROM bigTable2 x2 JOIN smallTable2 y2 ON (x2.key1 = y2.key1)) tmp1
      

      For this case, we will have three Map-only jobs (two for MapJoins and one for Union). It will be good to use a single Map-only job to execute this query.

      1. HIVE-4827.8.patch
        505 kB
        Yin Huai
      2. HIVE-4827.7.patch
        505 kB
        Yin Huai
      3. HIVE-4827.6.patch
        333 kB
        Yin Huai
      4. HIVE-4827.5.patch
        333 kB
        Yin Huai
      5. HIVE-4827.4.patch
        336 kB
        Yin Huai
      6. HIVE-4827.3.patch
        214 kB
        Yin Huai
      7. HIVE-4827.2.patch
        167 kB
        Yin Huai
      8. HIVE-4827.1.patch
        109 kB
        Yin Huai

        Issue Links

          Activity

          Yin Huai created issue -
          Yin Huai made changes -
          Field Original Value New Value
          Link This issue relates to HIVE-2206 [ HIVE-2206 ]
          Yin Huai made changes -
          Link This issue relates to HIVE-3952 [ HIVE-3952 ]
          Hide
          Yin Huai added a comment -

          I have been studying the code related to MapJoinOperator and CommonJoinResolver. Besides CommonJoinResolver, seems that we also need to change MapJoinOperator to load correct hash tables because some hash tables may not be needed by a MapJoinOperator.

          Show
          Yin Huai added a comment - I have been studying the code related to MapJoinOperator and CommonJoinResolver. Besides CommonJoinResolver, seems that we also need to change MapJoinOperator to load correct hash tables because some hash tables may not be needed by a MapJoinOperator.
          Yin Huai made changes -
          Description When hive.optimize.mapjoin.mapreduce is on, CommonJoinResolver can attach a Map-only job (MapJoin) to its following MapReduce job. But this merge only happens when the MapReduce job has a single input. With Correlation Optimizer (HIVE-2206), it is possible that the MapReduce job can have multiple inputs (for multiple operation paths). It is desired to improve CommonJoinResolver to merge a Map-only job to the corresponding Map task of the MapReduce job.

          Example:
          {code:sql}
          SELECT tmp1.key, count(*)
          FROM (SELECT x1.key2 AS key
                FROM bigTable1 x1 JOIN smallTable1 y1 ON (x1.key1 = y1.key1)
                GROUP BY x1.key2) tmp1
          JOIN (SELECT x2.key2 AS key
                FROM bigTable2 x2 JOIN smallTable2 y2 ON (x2.key1 = y2.key1)
                GROUP BY x2.key2) tmp2
          ON (tmp1.key = tmp2.key)
          GROUP BY tmp1.key;
          {\code}
          In this query, join operations inside tmp1 and tmp2 will be converted to two MapJoins. With Correlation Optimizer, aggregations in tmp1, tmp2, and join of tmp1 and tmp2, and the last aggregation will be executed in the same MapReduce job (Reduce side). Since this MapReduce job has two inputs, right now, CommonJoinResolver cannot attach two MapJoins to the Map side of a MapReduce job.
          When hive.optimize.mapjoin.mapreduce is on, CommonJoinResolver can attach a Map-only job (MapJoin) to its following MapReduce job. But this merge only happens when the MapReduce job has a single input. With Correlation Optimizer (HIVE-2206), it is possible that the MapReduce job can have multiple inputs (for multiple operation paths). It is desired to improve CommonJoinResolver to merge a Map-only job to the corresponding Map task of the MapReduce job.

          Example:
          {code:sql}
          SELECT tmp1.key, count(*)
          FROM (SELECT x1.key2 AS key
                FROM bigTable1 x1 JOIN smallTable1 y1 ON (x1.key1 = y1.key1)
                GROUP BY x1.key2) tmp1
          JOIN (SELECT x2.key2 AS key
                FROM bigTable2 x2 JOIN smallTable2 y2 ON (x2.key1 = y2.key1)
                GROUP BY x2.key2) tmp2
          ON (tmp1.key = tmp2.key)
          GROUP BY tmp1.key;
          {\code}
          In this query, join operations inside tmp1 and tmp2 will be converted to two MapJoins. With Correlation Optimizer, aggregations in tmp1, tmp2, and join of tmp1 and tmp2, and the last aggregation will be executed in the same MapReduce job (Reduce side). Since this MapReduce job has two inputs, right now, CommonJoinResolver cannot attach two MapJoins to the Map side of a MapReduce job.

          Another example:
          {code:sql}
          SELECT tmp1.key
          FROM (SELECT x1.key2 AS key
                FROM bigTable1 x1 JOIN smallTable1 y1 ON (x1.key1 = y1.key1)
                UNION ALL
                SELECT x2.key2 AS key
                FROM bigTable2 x2 JOIN smallTable2 y2 ON (x2.key1 = y2.key1)) tmp1
          {\code}
          For this case, we will have three Map-only jobs (two for MapJoins and one for Union). It will be good to use a single Map-only job to execute this query.
          Yin Huai made changes -
          Affects Version/s 0.12.0 [ 12324312 ]
          Yin Huai made changes -
          Component/s Query Processor [ 12312586 ]
          Yin Huai made changes -
          Description When hive.optimize.mapjoin.mapreduce is on, CommonJoinResolver can attach a Map-only job (MapJoin) to its following MapReduce job. But this merge only happens when the MapReduce job has a single input. With Correlation Optimizer (HIVE-2206), it is possible that the MapReduce job can have multiple inputs (for multiple operation paths). It is desired to improve CommonJoinResolver to merge a Map-only job to the corresponding Map task of the MapReduce job.

          Example:
          {code:sql}
          SELECT tmp1.key, count(*)
          FROM (SELECT x1.key2 AS key
                FROM bigTable1 x1 JOIN smallTable1 y1 ON (x1.key1 = y1.key1)
                GROUP BY x1.key2) tmp1
          JOIN (SELECT x2.key2 AS key
                FROM bigTable2 x2 JOIN smallTable2 y2 ON (x2.key1 = y2.key1)
                GROUP BY x2.key2) tmp2
          ON (tmp1.key = tmp2.key)
          GROUP BY tmp1.key;
          {\code}
          In this query, join operations inside tmp1 and tmp2 will be converted to two MapJoins. With Correlation Optimizer, aggregations in tmp1, tmp2, and join of tmp1 and tmp2, and the last aggregation will be executed in the same MapReduce job (Reduce side). Since this MapReduce job has two inputs, right now, CommonJoinResolver cannot attach two MapJoins to the Map side of a MapReduce job.

          Another example:
          {code:sql}
          SELECT tmp1.key
          FROM (SELECT x1.key2 AS key
                FROM bigTable1 x1 JOIN smallTable1 y1 ON (x1.key1 = y1.key1)
                UNION ALL
                SELECT x2.key2 AS key
                FROM bigTable2 x2 JOIN smallTable2 y2 ON (x2.key1 = y2.key1)) tmp1
          {\code}
          For this case, we will have three Map-only jobs (two for MapJoins and one for Union). It will be good to use a single Map-only job to execute this query.
          When hive.optimize.mapjoin.mapreduce is on, CommonJoinResolver can attach a Map-only job (MapJoin) to its following MapReduce job. But this merge only happens when the MapReduce job has a single input. With Correlation Optimizer (HIVE-2206), it is possible that the MapReduce job can have multiple inputs (for multiple operation paths). It is desired to improve CommonJoinResolver to merge a Map-only job to the corresponding Map task of the MapReduce job.

          Example:
          {code:sql}
          set hive.optimize.correlation=true;
          SELECT tmp1.key, count(*)
          FROM (SELECT x1.key2 AS key
                FROM bigTable1 x1 JOIN smallTable1 y1 ON (x1.key1 = y1.key1)
                GROUP BY x1.key2) tmp1
          JOIN (SELECT x2.key2 AS key
                FROM bigTable2 x2 JOIN smallTable2 y2 ON (x2.key1 = y2.key1)
                GROUP BY x2.key2) tmp2
          ON (tmp1.key = tmp2.key)
          GROUP BY tmp1.key;
          {\code}
          In this query, join operations inside tmp1 and tmp2 will be converted to two MapJoins. With Correlation Optimizer, aggregations in tmp1, tmp2, and join of tmp1 and tmp2, and the last aggregation will be executed in the same MapReduce job (Reduce side). Since this MapReduce job has two inputs, right now, CommonJoinResolver cannot attach two MapJoins to the Map side of a MapReduce job.

          Another example:
          {code:sql}
          SELECT tmp1.key
          FROM (SELECT x1.key2 AS key
                FROM bigTable1 x1 JOIN smallTable1 y1 ON (x1.key1 = y1.key1)
                UNION ALL
                SELECT x2.key2 AS key
                FROM bigTable2 x2 JOIN smallTable2 y2 ON (x2.key1 = y2.key1)) tmp1
          {\code}
          For this case, we will have three Map-only jobs (two for MapJoins and one for Union). It will be good to use a single Map-only job to execute this query.
          Yin Huai made changes -
          Description When hive.optimize.mapjoin.mapreduce is on, CommonJoinResolver can attach a Map-only job (MapJoin) to its following MapReduce job. But this merge only happens when the MapReduce job has a single input. With Correlation Optimizer (HIVE-2206), it is possible that the MapReduce job can have multiple inputs (for multiple operation paths). It is desired to improve CommonJoinResolver to merge a Map-only job to the corresponding Map task of the MapReduce job.

          Example:
          {code:sql}
          set hive.optimize.correlation=true;
          SELECT tmp1.key, count(*)
          FROM (SELECT x1.key2 AS key
                FROM bigTable1 x1 JOIN smallTable1 y1 ON (x1.key1 = y1.key1)
                GROUP BY x1.key2) tmp1
          JOIN (SELECT x2.key2 AS key
                FROM bigTable2 x2 JOIN smallTable2 y2 ON (x2.key1 = y2.key1)
                GROUP BY x2.key2) tmp2
          ON (tmp1.key = tmp2.key)
          GROUP BY tmp1.key;
          {\code}
          In this query, join operations inside tmp1 and tmp2 will be converted to two MapJoins. With Correlation Optimizer, aggregations in tmp1, tmp2, and join of tmp1 and tmp2, and the last aggregation will be executed in the same MapReduce job (Reduce side). Since this MapReduce job has two inputs, right now, CommonJoinResolver cannot attach two MapJoins to the Map side of a MapReduce job.

          Another example:
          {code:sql}
          SELECT tmp1.key
          FROM (SELECT x1.key2 AS key
                FROM bigTable1 x1 JOIN smallTable1 y1 ON (x1.key1 = y1.key1)
                UNION ALL
                SELECT x2.key2 AS key
                FROM bigTable2 x2 JOIN smallTable2 y2 ON (x2.key1 = y2.key1)) tmp1
          {\code}
          For this case, we will have three Map-only jobs (two for MapJoins and one for Union). It will be good to use a single Map-only job to execute this query.
          When hive.optimize.mapjoin.mapreduce is on, CommonJoinResolver can attach a Map-only job (MapJoin) to its following MapReduce job. But this merge only happens when the MapReduce job has a single input. With Correlation Optimizer (HIVE-2206), it is possible that the MapReduce job can have multiple inputs (for multiple operation paths). It is desired to improve CommonJoinResolver to merge a Map-only job to the corresponding Map task of the MapReduce job.

          Example:
          {code:sql}
          set hive.optimize.correlation=true;
          set hive.auto.convert.join=true;
          set hive.optimize.mapjoin.mapreduce=true;
          SELECT tmp1.key, count(*)
          FROM (SELECT x1.key2 AS key
                FROM bigTable1 x1 JOIN smallTable1 y1 ON (x1.key1 = y1.key1)
                GROUP BY x1.key2) tmp1
          JOIN (SELECT x2.key2 AS key
                FROM bigTable2 x2 JOIN smallTable2 y2 ON (x2.key1 = y2.key1)
                GROUP BY x2.key2) tmp2
          ON (tmp1.key = tmp2.key)
          GROUP BY tmp1.key;
          {\code}
          In this query, join operations inside tmp1 and tmp2 will be converted to two MapJoins. With Correlation Optimizer, aggregations in tmp1, tmp2, and join of tmp1 and tmp2, and the last aggregation will be executed in the same MapReduce job (Reduce side). Since this MapReduce job has two inputs, right now, CommonJoinResolver cannot attach two MapJoins to the Map side of a MapReduce job.

          Another example:
          {code:sql}
          SELECT tmp1.key
          FROM (SELECT x1.key2 AS key
                FROM bigTable1 x1 JOIN smallTable1 y1 ON (x1.key1 = y1.key1)
                UNION ALL
                SELECT x2.key2 AS key
                FROM bigTable2 x2 JOIN smallTable2 y2 ON (x2.key1 = y2.key1)) tmp1
          {\code}
          For this case, we will have three Map-only jobs (two for MapJoins and one for Union). It will be good to use a single Map-only job to execute this query.
          Hide
          Yin Huai added a comment -

          Actually, I only need to change CommonJoinTaskDispatcher.mergeMapJoinTaskWithMapReduceTask. No need to change MapJoinOperator. Just make my patch work for a simple union query shown in description. Should be able to have a patch uploaded soon.

          Show
          Yin Huai added a comment - Actually, I only need to change CommonJoinTaskDispatcher.mergeMapJoinTaskWithMapReduceTask. No need to change MapJoinOperator. Just make my patch work for a simple union query shown in description. Should be able to have a patch uploaded soon.
          Yin Huai made changes -
          Description When hive.optimize.mapjoin.mapreduce is on, CommonJoinResolver can attach a Map-only job (MapJoin) to its following MapReduce job. But this merge only happens when the MapReduce job has a single input. With Correlation Optimizer (HIVE-2206), it is possible that the MapReduce job can have multiple inputs (for multiple operation paths). It is desired to improve CommonJoinResolver to merge a Map-only job to the corresponding Map task of the MapReduce job.

          Example:
          {code:sql}
          set hive.optimize.correlation=true;
          set hive.auto.convert.join=true;
          set hive.optimize.mapjoin.mapreduce=true;
          SELECT tmp1.key, count(*)
          FROM (SELECT x1.key2 AS key
                FROM bigTable1 x1 JOIN smallTable1 y1 ON (x1.key1 = y1.key1)
                GROUP BY x1.key2) tmp1
          JOIN (SELECT x2.key2 AS key
                FROM bigTable2 x2 JOIN smallTable2 y2 ON (x2.key1 = y2.key1)
                GROUP BY x2.key2) tmp2
          ON (tmp1.key = tmp2.key)
          GROUP BY tmp1.key;
          {\code}
          In this query, join operations inside tmp1 and tmp2 will be converted to two MapJoins. With Correlation Optimizer, aggregations in tmp1, tmp2, and join of tmp1 and tmp2, and the last aggregation will be executed in the same MapReduce job (Reduce side). Since this MapReduce job has two inputs, right now, CommonJoinResolver cannot attach two MapJoins to the Map side of a MapReduce job.

          Another example:
          {code:sql}
          SELECT tmp1.key
          FROM (SELECT x1.key2 AS key
                FROM bigTable1 x1 JOIN smallTable1 y1 ON (x1.key1 = y1.key1)
                UNION ALL
                SELECT x2.key2 AS key
                FROM bigTable2 x2 JOIN smallTable2 y2 ON (x2.key1 = y2.key1)) tmp1
          {\code}
          For this case, we will have three Map-only jobs (two for MapJoins and one for Union). It will be good to use a single Map-only job to execute this query.
          When hive.optimize.mapjoin.mapreduce is on, CommonJoinResolver can attach a Map-only job (MapJoin) to its following MapReduce job. But this merge only happens when the MapReduce job has a single input. With Correlation Optimizer (HIVE-2206), it is possible that the MapReduce job can have multiple inputs (for multiple operation paths). It is desired to improve CommonJoinResolver to merge a Map-only job to the corresponding Map task of the MapReduce job.

          Example:
          {code:sql}
          set hive.optimize.correlation=true;
          set hive.auto.convert.join=true;
          set hive.optimize.mapjoin.mapreduce=true;
          SELECT tmp1.key, count(*)
          FROM (SELECT x1.key1 AS key
                FROM bigTable1 x1 JOIN smallTable1 y1 ON (x1.key1 = y1.key1)
                GROUP BY x1.key1) tmp1
          JOIN (SELECT x2.key2 AS key
                FROM bigTable2 x2 JOIN smallTable2 y2 ON (x2.key1 = y2.key1)
                GROUP BY x2.key2) tmp2
          ON (tmp1.key = tmp2.key)
          GROUP BY tmp1.key;
          {\code}
          In this query, join operations inside tmp1 and tmp2 will be converted to two MapJoins. With Correlation Optimizer, aggregations in tmp1, tmp2, and join of tmp1 and tmp2, and the last aggregation will be executed in the same MapReduce job (Reduce side). Since this MapReduce job has two inputs, right now, CommonJoinResolver cannot attach two MapJoins to the Map side of a MapReduce job.

          Another example:
          {code:sql}
          SELECT tmp1.key
          FROM (SELECT x1.key2 AS key
                FROM bigTable1 x1 JOIN smallTable1 y1 ON (x1.key1 = y1.key1)
                UNION ALL
                SELECT x2.key2 AS key
                FROM bigTable2 x2 JOIN smallTable2 y2 ON (x2.key1 = y2.key1)) tmp1
          {\code}
          For this case, we will have three Map-only jobs (two for MapJoins and one for Union). It will be good to use a single Map-only job to execute this query.
          Yin Huai made changes -
          Description When hive.optimize.mapjoin.mapreduce is on, CommonJoinResolver can attach a Map-only job (MapJoin) to its following MapReduce job. But this merge only happens when the MapReduce job has a single input. With Correlation Optimizer (HIVE-2206), it is possible that the MapReduce job can have multiple inputs (for multiple operation paths). It is desired to improve CommonJoinResolver to merge a Map-only job to the corresponding Map task of the MapReduce job.

          Example:
          {code:sql}
          set hive.optimize.correlation=true;
          set hive.auto.convert.join=true;
          set hive.optimize.mapjoin.mapreduce=true;
          SELECT tmp1.key, count(*)
          FROM (SELECT x1.key1 AS key
                FROM bigTable1 x1 JOIN smallTable1 y1 ON (x1.key1 = y1.key1)
                GROUP BY x1.key1) tmp1
          JOIN (SELECT x2.key2 AS key
                FROM bigTable2 x2 JOIN smallTable2 y2 ON (x2.key1 = y2.key1)
                GROUP BY x2.key2) tmp2
          ON (tmp1.key = tmp2.key)
          GROUP BY tmp1.key;
          {\code}
          In this query, join operations inside tmp1 and tmp2 will be converted to two MapJoins. With Correlation Optimizer, aggregations in tmp1, tmp2, and join of tmp1 and tmp2, and the last aggregation will be executed in the same MapReduce job (Reduce side). Since this MapReduce job has two inputs, right now, CommonJoinResolver cannot attach two MapJoins to the Map side of a MapReduce job.

          Another example:
          {code:sql}
          SELECT tmp1.key
          FROM (SELECT x1.key2 AS key
                FROM bigTable1 x1 JOIN smallTable1 y1 ON (x1.key1 = y1.key1)
                UNION ALL
                SELECT x2.key2 AS key
                FROM bigTable2 x2 JOIN smallTable2 y2 ON (x2.key1 = y2.key1)) tmp1
          {\code}
          For this case, we will have three Map-only jobs (two for MapJoins and one for Union). It will be good to use a single Map-only job to execute this query.
          When hive.optimize.mapjoin.mapreduce is on, CommonJoinResolver can attach a Map-only job (MapJoin) to its following MapReduce job. But this merge only happens when the MapReduce job has a single input. With Correlation Optimizer (HIVE-2206), it is possible that the MapReduce job can have multiple inputs (for multiple operation paths). It is desired to improve CommonJoinResolver to merge a Map-only job to the corresponding Map task of the MapReduce job.

          Example:
          {code:sql}
          set hive.optimize.correlation=true;
          set hive.auto.convert.join=true;
          set hive.optimize.mapjoin.mapreduce=true;
          SELECT tmp1.key, count(*)
          FROM (SELECT x1.key1 AS key
                FROM bigTable1 x1 JOIN smallTable1 y1 ON (x1.key1 = y1.key1)
                GROUP BY x1.key1) tmp1
          JOIN (SELECT x2.key2 AS key
                FROM bigTable2 x2 JOIN smallTable2 y2 ON (x2.key2 = y2.key2)
                GROUP BY x2.key2) tmp2
          ON (tmp1.key = tmp2.key)
          GROUP BY tmp1.key;
          {\code}
          In this query, join operations inside tmp1 and tmp2 will be converted to two MapJoins. With Correlation Optimizer, aggregations in tmp1, tmp2, and join of tmp1 and tmp2, and the last aggregation will be executed in the same MapReduce job (Reduce side). Since this MapReduce job has two inputs, right now, CommonJoinResolver cannot attach two MapJoins to the Map side of a MapReduce job.

          Another example:
          {code:sql}
          SELECT tmp1.key
          FROM (SELECT x1.key2 AS key
                FROM bigTable1 x1 JOIN smallTable1 y1 ON (x1.key1 = y1.key1)
                UNION ALL
                SELECT x2.key2 AS key
                FROM bigTable2 x2 JOIN smallTable2 y2 ON (x2.key1 = y2.key1)) tmp1
          {\code}
          For this case, we will have three Map-only jobs (two for MapJoins and one for Union). It will be good to use a single Map-only job to execute this query.
          Yin Huai made changes -
          Remote Link This issue links to "Review board (Web Link)" [ 12429 ]
          Hide
          Yin Huai added a comment -

          Add my first draft patch.

          Show
          Yin Huai added a comment - Add my first draft patch.
          Yin Huai made changes -
          Attachment HIVE-4827.1.patch [ 12593386 ]
          Hide
          Yin Huai added a comment -

          I am running tests. Will see if there is any result which needs to be updated or any bug in my patch.

          Show
          Yin Huai added a comment - I am running tests. Will see if there is any result which needs to be updated or any bug in my patch.
          Hide
          Edward Capriolo added a comment -

          Because two map reduce jobs are becoming a single one now does this mean that there is a greater chance of the map task failing with oom conditions?

          Show
          Edward Capriolo added a comment - Because two map reduce jobs are becoming a single one now does this mean that there is a greater chance of the map task failing with oom conditions?
          Hide
          Yin Huai added a comment -

          Hi Edward, let me explain with an example.

          Let's say we have a query like the one shown in the description...

          SELECT tmp1.key
          FROM (SELECT x1.key2 AS key
                FROM bigTable1 x1 JOIN smallTable1 y1 ON (x1.key1 = y1.key1)
                UNION ALL
                SELECT x2.key2 AS key
                FROM bigTable2 x2 JOIN smallTable2 y2 ON (x2.key1 = y2.key1)) tmp1
          

          With this patch, we will just need a single Map-only job to evaluate it. Since we have two big tables (bigTable1 and bigTable2), in every Map task, only a single operation path will be initialized. Some tasks will process the MapJoin of bigTable1 and smallTable1, and some other tasks will process the MapJoin of bigTable2 and smallTable2. In this case, I think we will not increase the memory footprint of a Map task.

          When we slightly change this query to ...

          SELECT tmp1.key
          FROM (SELECT x1.key2 AS key
                FROM bigTable1 x1 JOIN smallTable1 y1 ON (x1.key1 = y1.key1)
                UNION ALL
                SELECT x2.key2 AS key
                FROM bigTable1 x2 JOIN smallTable2 y2 ON (x2.key1 = y2.key1)) tmp1
          

          because we are using a single big table (bigTable1), we can have multiple aliases associated to a single input path. In this case, multiple TableScanOperators will be connected to a single MapOperator because we will just scan the table once. Because we evaluate both the MapJoin of bigTable1 and smallTable1, and MapJoin of bigTable1 and smallTable2 in the same Map task, this task has a higher memory footprint than the Map task which only process a single MapJoin (because we will load more Hash Tables). I think we still want to use a single Map-only job to evaluate this query because we can have shared scan, and we do not need to materialize intermediate results and load them back again just for UNION ALL. Seems HIVE-3996 is related. I will check it and add the logic to enforce the memory limit when we have shared scan.

          Show
          Yin Huai added a comment - Hi Edward, let me explain with an example. Let's say we have a query like the one shown in the description... SELECT tmp1.key FROM (SELECT x1.key2 AS key FROM bigTable1 x1 JOIN smallTable1 y1 ON (x1.key1 = y1.key1) UNION ALL SELECT x2.key2 AS key FROM bigTable2 x2 JOIN smallTable2 y2 ON (x2.key1 = y2.key1)) tmp1 With this patch, we will just need a single Map-only job to evaluate it. Since we have two big tables (bigTable1 and bigTable2), in every Map task, only a single operation path will be initialized. Some tasks will process the MapJoin of bigTable1 and smallTable1, and some other tasks will process the MapJoin of bigTable2 and smallTable2. In this case, I think we will not increase the memory footprint of a Map task. When we slightly change this query to ... SELECT tmp1.key FROM (SELECT x1.key2 AS key FROM bigTable1 x1 JOIN smallTable1 y1 ON (x1.key1 = y1.key1) UNION ALL SELECT x2.key2 AS key FROM bigTable1 x2 JOIN smallTable2 y2 ON (x2.key1 = y2.key1)) tmp1 because we are using a single big table (bigTable1), we can have multiple aliases associated to a single input path. In this case, multiple TableScanOperators will be connected to a single MapOperator because we will just scan the table once. Because we evaluate both the MapJoin of bigTable1 and smallTable1, and MapJoin of bigTable1 and smallTable2 in the same Map task, this task has a higher memory footprint than the Map task which only process a single MapJoin (because we will load more Hash Tables). I think we still want to use a single Map-only job to evaluate this query because we can have shared scan, and we do not need to materialize intermediate results and load them back again just for UNION ALL. Seems HIVE-3996 is related. I will check it and add the logic to enforce the memory limit when we have shared scan.
          Hide
          Yin Huai added a comment -

          Addressed Edward's comment and update test results.

          Show
          Yin Huai added a comment - Addressed Edward's comment and update test results.
          Yin Huai made changes -
          Attachment HIVE-4827.2.patch [ 12593451 ]
          Yin Huai made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Yin Huai added a comment -
          Show
          Yin Huai added a comment - Review board: https://reviews.apache.org/r/12795/
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12593451/HIVE-4827.2.patch

          ERROR: -1 due to 1 failed/errored test(s), 2647 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin6
          

          Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/123/testReport
          Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/123/console

          Messages:

          Executing org.apache.hive.ptest.execution.CleanupPhase
          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests failed with: TestsFailedException: 1 tests failed
          

          This message is automatically generated.

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12593451/HIVE-4827.2.patch ERROR: -1 due to 1 failed/errored test(s), 2647 tests executed Failed tests: org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin6 Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/123/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/123/console Messages: Executing org.apache.hive.ptest.execution.CleanupPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests failed with: TestsFailedException: 1 tests failed This message is automatically generated.
          Yin Huai made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Yin Huai added a comment -

          Cannot reproduce the failed test case (TestMinimrCliDriver.testCliDriver_bucketmapjoin6) in my laptop. I will use another machine to test it.

          Show
          Yin Huai added a comment - Cannot reproduce the failed test case (TestMinimrCliDriver.testCliDriver_bucketmapjoin6) in my laptop. I will use another machine to test it.
          Hide
          Brock Noland added a comment -

          Yeah that test is flaky. I just submitted another run of the precommit.

          Show
          Brock Noland added a comment - Yeah that test is flaky. I just submitted another run of the precommit.
          Hide
          Yin Huai added a comment -

          Thanks!

          Show
          Yin Huai added a comment - Thanks!
          Hide
          Yin Huai added a comment -

          Found an issue related to partitioned big tables. Will fix the issue later.

          Show
          Yin Huai added a comment - Found an issue related to partitioned big tables. Will fix the issue later.
          Hide
          Gunther Hagleitner added a comment -

          Comments on RB. Nice optimization.

          Show
          Gunther Hagleitner added a comment - Comments on RB. Nice optimization.
          Yin Huai made changes -
          Link This issue is related to HIVE-4927 [ HIVE-4927 ]
          Hide
          Yin Huai added a comment -

          With this work, we can merge mergeMapJoinTaskWithChildMapJoinTask with mergeMapJoinTaskWithMapReduceTask and drop the flag of 'hive.optimize.mapjoin.mapreduce'.

          Show
          Yin Huai added a comment - With this work, we can merge mergeMapJoinTaskWithChildMapJoinTask with mergeMapJoinTaskWithMapReduceTask and drop the flag of 'hive.optimize.mapjoin.mapreduce'.
          Hide
          Yin Huai added a comment -

          Also, with this work, different Map tasks (operation paths) can load different set of hash tables. So, we should check the size of those hash tables per operation path.

          Show
          Yin Huai added a comment - Also, with this work, different Map tasks (operation paths) can load different set of hash tables. So, we should check the size of those hash tables per operation path.
          Yin Huai made changes -
          Attachment HIVE-4827.3.patch [ 12594282 ]
          Hide
          Yin Huai added a comment -

          .3 is not for review. I am currently running unit tests.

          Show
          Yin Huai added a comment - .3 is not for review. I am currently running unit tests.
          Yin Huai made changes -
          Attachment HIVE-4827.4.patch [ 12594435 ]
          Yin Huai made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Yin Huai added a comment -

          Updated patch has been uploaded to RB. Mark it as PA to trigger precommit tests.

          Show
          Yin Huai added a comment - Updated patch has been uploaded to RB. Mark it as PA to trigger precommit tests.
          Hide
          Hive QA added a comment -

          Overall: -1 no tests executed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12594435/HIVE-4827.4.patch

          Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/199/testReport
          Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/199/console

          Messages:

          Executing org.apache.hive.ptest.execution.CleanupPhase
          Executing org.apache.hive.ptest.execution.PrepPhase
          Tests failed with: NonZeroExitCodeException: Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ [[ -n '' ]]
          + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128'
          + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128'
          + cd /data/hive-ptest/working/
          + tee /data/hive-ptest/logs/PreCommit-HIVE-Build-199/source-prep.txt
          + mkdir -p maven ivy
          + [[ svn = \s\v\n ]]
          + [[ -n '' ]]
          + [[ -d apache-svn-trunk-source ]]
          + [[ ! -d apache-svn-trunk-source/.svn ]]
          + [[ ! -d apache-svn-trunk-source ]]
          + cd apache-svn-trunk-source
          + svn revert -R .
          Reverted 'contrib/src/test/results/clientpositive/serde_typedbytes4.q.out'
          Reverted 'contrib/src/test/results/clientpositive/dboutput.q.out'
          Reverted 'ql/src/test/results/clientpositive/auto_join27.q.out'
          Reverted 'ql/src/test/results/clientpositive/join14_hadoop20.q.out'
          Reverted 'ql/src/test/results/clientpositive/udf9.q.out'
          Reverted 'ql/src/test/results/clientpositive/ppd_clusterby.q.out'
          Reverted 'ql/src/test/results/clientpositive/auto_join13.q.out'
          Reverted 'ql/src/test/results/clientpositive/groupby_multi_single_reducer2.q.out'
          Reverted 'ql/src/test/results/clientpositive/index_auto_update.q.out'
          Reverted 'ql/src/test/results/clientpositive/notable_alias1.q.out'
          Reverted 'ql/src/test/results/clientpositive/load_dyn_part13.q.out'
          Reverted 'ql/src/test/results/clientpositive/udf_union.q.out'
          Reverted 'ql/src/test/results/clientpositive/sort_merge_join_desc_3.q.out'
          Reverted 'ql/src/test/results/clientpositive/join38.q.out'
          Reverted 'ql/src/test/results/clientpositive/input_part7.q.out'
          Reverted 'ql/src/test/results/clientpositive/groupby_grouping_sets4.q.out'
          Reverted 'ql/src/test/results/clientpositive/pcr.q.out'
          Reverted 'ql/src/test/results/clientpositive/mapjoin_filter_on_outerjoin.q.out'
          Reverted 'ql/src/test/results/clientpositive/index_auto_partitioned.q.out'
          Reverted 'ql/src/test/results/clientpositive/cluster.q.out'
          Reverted 'ql/src/test/results/clientpositive/input12_hadoop20.q.out'
          Reverted 'ql/src/test/results/clientpositive/index_auto_file_format.q.out'
          Reverted 'ql/src/test/results/clientpositive/input_part2.q.out'
          Reverted 'ql/src/test/results/clientpositive/ppd_transform.q.out'
          Reverted 'ql/src/test/results/clientpositive/create_view.q.out'
          Reverted 'ql/src/test/results/clientpositive/alter_partition_coltype.q.out'
          Reverted 'ql/src/test/results/clientpositive/infer_const_type.q.out'
          Reverted 'ql/src/test/results/clientpositive/correlationoptimizer8.q.out'
          Reverted 'ql/src/test/results/clientpositive/join7.q.out'
          Reverted 'ql/src/test/results/clientpositive/auto_join4.q.out'
          Reverted 'ql/src/test/results/clientpositive/input2_limit.q.out'
          Reverted 'ql/src/test/results/clientpositive/input11_limit.q.out'
          Reverted 'ql/src/test/results/clientpositive/udf_like.q.out'
          Reverted 'ql/src/test/results/clientpositive/udf_hour.q.out'
          Reverted 'ql/src/test/results/clientpositive/groupby_sort_1.q.out'
          Reverted 'ql/src/test/results/clientpositive/ppr_pushdown3.q.out'
          Reverted 'ql/src/test/results/clientpositive/input11.q.out'
          Reverted 'ql/src/test/results/clientpositive/noalias_subq1.q.out'
          Reverted 'ql/src/test/results/clientpositive/auto_join29.q.out'
          Reverted 'ql/src/test/results/clientpositive/index_auto_mult_tables_compact.q.out'
          Reverted 'ql/src/test/results/clientpositive/regex_col.q.out'
          Reverted 'ql/src/test/results/clientpositive/allcolref_in_udf.q.out'
          Reverted 'ql/src/test/results/clientpositive/udf1.q.out'
          Reverted 'ql/src/test/results/clientpositive/join12.q.out'
          Reverted 'ql/src/test/results/clientpositive/join35.q.out'
          Reverted 'ql/src/test/results/clientpositive/input_part4.q.out'
          Reverted 'ql/src/test/results/clientpositive/union20.q.out'
          Reverted 'ql/src/test/results/clientpositive/column_access_stats.q.out'
          Reverted 'ql/src/test/results/clientpositive/join21.q.out'
          Reverted 'ql/src/test/results/clientpositive/ppd_multi_insert.q.out'
          Reverted 'ql/src/test/results/clientpositive/index_stale_partitioned.q.out'
          Reverted 'ql/src/test/results/clientpositive/auto_join6.q.out'
          Reverted 'ql/src/test/results/clientpositive/correlationoptimizer10.q.out'
          Reverted 'ql/src/test/results/clientpositive/udf_lower.q.out'
          Reverted 'ql/src/test/results/clientpositive/nullgroup.q.out'
          Reverted 'ql/src/test/results/clientpositive/join4.q.out'
          Reverted 'ql/src/test/results/clientpositive/nullgroup4_multi_distinct.q.out'
          Reverted 'ql/src/test/results/clientpositive/index_bitmap_compression.q.out'
          Reverted 'ql/src/test/results/clientpositive/input18.q.out'
          Reverted 'ql/src/test/results/clientpositive/multi_insert_gby.q.out'
          Reverted 'ql/src/test/results/clientpositive/groupby_multi_single_reducer.q.out'
          Reverted 'ql/src/test/results/clientpositive/index_auto_mult_tables.q.out'
          Reverted 'ql/src/test/results/clientpositive/rand_partitionpruner3.q.out'
          Reverted 'ql/src/test/results/clientpositive/input13.q.out'
          Reverted 'ql/src/test/results/clientpositive/order2.q.out'
          Reverted 'ql/src/test/results/clientpositive/multi_insert.q.out'
          Reverted 'ql/src/test/results/clientpositive/ppd_udf_col.q.out'
          Reverted 'ql/src/test/results/clientpositive/auto_join12.q.out'
          Reverted 'ql/src/test/results/clientpositive/ppd_outer_join1.q.out'
          Reverted 'ql/src/test/results/clientpositive/ppd2.q.out'
          Reverted 'ql/src/test/results/clientpositive/index_auto_empty.q.out'
          Reverted 'ql/src/test/results/clientpositive/auto_join21.q.out'
          Reverted 'ql/src/test/results/clientpositive/sort_merge_join_desc_2.q.out'
          Reverted 'ql/src/test/results/clientpositive/union22.q.out'
          Reverted 'ql/src/test/results/clientpositive/join23.q.out'
          Reverted 'ql/src/test/results/clientpositive/udf_second.q.out'
          Reverted 'ql/src/test/results/clientpositive/index_auto_multiple.q.out'
          Reverted 'ql/src/test/results/clientpositive/union.q.out'
          Reverted 'ql/src/test/results/clientpositive/udf_10_trims.q.out'
          Reverted 'ql/src/test/results/clientpositive/multi_insert_move_tasks_share_dependencies.q.out'
          Reverted 'ql/src/test/results/clientpositive/input_part1.q.out'
          Reverted 'ql/src/test/results/clientpositive/groupby_sort_skew_1.q.out'
          Reverted 'ql/src/test/results/clientpositive/auto_join8.q.out'
          Reverted 'ql/src/test/results/clientpositive/index_compression.q.out'
          Reverted 'ql/src/test/results/clientpositive/udf_parse_url.q.out'
          Reverted 'ql/src/test/results/clientpositive/join41.q.out'
          Reverted 'ql/src/test/results/clientpositive/multi_insert_lateral_view.q.out'
          Reverted 'ql/src/test/results/clientpositive/no_hooks.q.out'
          Reverted 'ql/src/test/results/clientpositive/join6.q.out'
          Reverted 'ql/src/test/results/clientpositive/index_bitmap3.q.out'
          Reverted 'ql/src/test/results/clientpositive/groupby_position.q.out'
          Reverted 'ql/src/test/results/clientpositive/transform_ppr2.q.out'
          Reverted 'ql/src/test/results/clientpositive/set_processor_namespaces.q.out'
          Reverted 'ql/src/test/results/clientpositive/union_ppr.q.out'
          Reverted 'ql/src/test/results/clientpositive/auto_join14_hadoop20.q.out'
          Reverted 'ql/src/test/results/clientpositive/index_bitmap_auto_partitioned.q.out'
          Reverted 'ql/src/test/results/clientpositive/nonmr_fetch.q.out'
          Reverted 'ql/src/test/results/clientpositive/router_join_ppr.q.out'
          Reverted 'ql/src/test/results/clientpositive/auto_join28.q.out'
          Reverted 'ql/src/test/results/clientpositive/groupby_multi_single_reducer3.q.out'
          Reverted 'ql/src/test/results/clientpositive/udf_minute.q.out'
          Reverted 'ql/src/test/results/clientpositive/input42.q.out'
          Reverted 'ql/src/test/results/clientpositive/having.q.out'
          Reverted 'ql/src/test/results/clientpositive/quote1.q.out'
          Reverted 'ql/src/test/results/clientpositive/notable_alias2.q.out'
          Reverted 'ql/src/test/results/clientpositive/index_auto_unused.q.out'
          Reverted 'ql/src/test/results/clientpositive/auto_join23.q.out'
          Reverted 'ql/src/test/results/clientpositive/join16.q.out'
          Reverted 'ql/src/test/results/clientpositive/sort_merge_join_desc_4.q.out'
          Reverted 'ql/src/test/results/clientpositive/nullgroup2.q.out'
          Reverted 'ql/src/test/results/clientpositive/join39.q.out'
          Reverted 'ql/src/test/results/clientpositive/union24.q.out'
          Reverted 'ql/src/test/results/clientpositive/index_auto.q.out'
          Reverted 'ql/src/test/results/clientpositive/rcfile_null_value.q.out'
          Reverted 'ql/src/test/results/clientpositive/union33.q.out'
          Reverted 'ql/src/test/results/clientpositive/join11.q.out'
          Reverted 'ql/src/test/results/clientpositive/join34.q.out'
          Reverted 'ql/src/test/results/clientpositive/join20.q.out'
          Reverted 'ql/src/test/results/clientpositive/join8.q.out'
          Reverted 'ql/src/test/results/clientpositive/subq.q.out'
          Reverted 'ql/src/test/results/clientpositive/auto_join5.q.out'
          Reverted 'ql/src/test/results/clientpositive/input14_limit.q.out'
          Reverted 'ql/src/test/results/clientpositive/louter_join_ppr.q.out'
          Reverted 'ql/src/test/results/clientpositive/auto_join0.q.out'
          Reverted 'ql/src/test/results/clientpositive/skewjoin.q.out'
          Reverted 'ql/src/test/results/clientpositive/index_auto_self_join.q.out'
          Reverted 'ql/src/test/results/clientpositive/auto_join16.q.out'
          Reverted 'ql/src/test/results/clientpositive/nullgroup4.q.out'
          Reverted 'ql/src/test/results/clientpositive/auto_join11.q.out'
          Reverted 'ql/src/test/results/clientpositive/subq2.q.out'
          Reverted 'ql/src/test/results/clientpositive/auto_join20.q.out'
          Reverted 'ql/src/test/results/clientpositive/sort_merge_join_desc_1.q.out'
          Reverted 'ql/src/test/results/clientpositive/join13.q.out'
          Reverted 'ql/src/test/results/clientpositive/input_part5.q.out'
          Reverted 'ql/src/test/results/clientpositive/cast1.q.out'
          Reverted 'ql/src/test/results/clientpositive/index_stale.q.out'
          Reverted 'ql/src/test/results/clientpositive/auto_join7.q.out'
          Reverted 'ql/src/test/results/clientpositive/join40.q.out'
          Reverted 'ql/src/test/results/clientpositive/join5.q.out'
          Reverted 'ql/src/test/results/clientpositive/outer_join_ppr.q.out'
          Reverted 'ql/src/test/results/clientpositive/input1_limit.q.out'
          Reverted 'ql/src/test/results/clientpositive/index_bitmap_auto.q.out'
          Reverted 'ql/src/test/results/clientpositive/transform_ppr1.q.out'
          Reverted 'ql/src/test/results/clientpositive/join0.q.out'
          Reverted 'ql/src/test/results/clientpositive/regexp_extract.q.out'
          Reverted 'ql/src/test/results/clientpositive/ctas_colname.q.out'
          Reverted 'ql/src/test/results/clientpositive/mergejoins.q.out'
          Reverted 'ql/src/test/results/clientpositive/input14.q.out'
          Reverted 'ql/src/test/results/beelinepositive/auto_join27.q.out'
          Reverted 'ql/src/test/results/beelinepositive/auto_join13.q.out'
          Reverted 'ql/src/test/results/beelinepositive/create_view.q.out'
          Reverted 'ql/src/test/results/beelinepositive/auto_join28.q.out'
          Reverted 'ql/src/test/results/beelinepositive/auto_join4.q.out'
          Reverted 'ql/src/test/results/beelinepositive/auto_join23.q.out'
          Reverted 'ql/src/test/results/beelinepositive/auto_join29.q.out'
          Reverted 'ql/src/test/results/beelinepositive/auto_join5.q.out'
          Reverted 'ql/src/test/results/beelinepositive/auto_join0.q.out'
          Reverted 'ql/src/test/results/beelinepositive/auto_join6.q.out'
          Reverted 'ql/src/test/results/beelinepositive/auto_join16.q.out'
          Reverted 'ql/src/test/results/beelinepositive/auto_join20.q.out'
          Reverted 'ql/src/test/results/beelinepositive/auto_join7.q.out'
          Reverted 'ql/src/test/results/beelinepositive/auto_join12.q.out'
          Reverted 'ql/src/test/results/beelinepositive/auto_join21.q.out'
          Reverted 'ql/src/test/results/beelinepositive/auto_join8.q.out'
          Reverted 'ql/src/test/results/compiler/plan/input2.q.xml'
          Reverted 'ql/src/test/results/compiler/plan/input3.q.xml'
          Reverted 'ql/src/test/results/compiler/plan/join4.q.xml'
          Reverted 'ql/src/test/results/compiler/plan/input4.q.xml'
          Reverted 'ql/src/test/results/compiler/plan/join5.q.xml'
          Reverted 'ql/src/test/results/compiler/plan/join6.q.xml'
          Reverted 'ql/src/test/results/compiler/plan/join7.q.xml'
          Reverted 'ql/src/test/results/compiler/plan/join8.q.xml'
          Reverted 'ql/src/test/results/compiler/plan/union.q.xml'
          Reverted 'ql/src/test/results/compiler/plan/udf1.q.xml'
          Reverted 'ql/src/test/results/compiler/plan/input_part1.q.xml'
          Reverted 'ql/src/test/results/compiler/plan/subq.q.xml'
          Reverted 'ql/src/test/results/compiler/plan/cast1.q.xml'
          Reverted 'ql/src/test/results/compiler/plan/input1.q.xml'
          Reverted 'ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java'
          ++ egrep -v '^X|^Performing status on external'
          ++ awk '{print $2}'
          ++ svn status --no-ignore
          + rm -rf build hcatalog/build hcatalog/core/build hcatalog/storage-handlers/hbase/build hcatalog/server-extensions/build hcatalog/webhcat/svr/build hcatalog/webhcat/java-client/build hcatalog/hcatalog-pig-adapter/build common/src/gen
          + svn update
          D    testutils/ptest2/src/test/resources/TEST-SomeTest-failure.xml
          D    testutils/ptest2/src/test/resources/test-outputs/TEST-skewjoin_union_remove_1.q-6fa31776-d2b0-4e13-9761-11f750627ad1-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml
          D    testutils/ptest2/src/test/resources/test-outputs/TEST-union_remove_9.q-acb9de8f-1b9c-4874-924c-b2107ca7b07c-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml
          D    testutils/ptest2/src/test/resources/test-outputs/TEST-skewjoin.q-ab8536a7-1b5c-45ed-ba29-14450f27db8b-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml
          D    testutils/ptest2/src/test/resources/test-outputs/TEST-index_auth.q-bucketcontex-ba31fb54-1d7f-4c70-a89d-477b7d155191-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml
          D    testutils/ptest2/src/test/resources/test-outputs/TEST-SomeTest-truncated.xml
          
          Fetching external item into 'hcatalog/src/test/e2e/harness'
          Updated external to revision 1507508.
          
          Updated to revision 1507508.
          + patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
          + patchFilePath=/data/hive-ptest/working/scratch/build.patch
          + [[ -f /data/hive-ptest/working/scratch/build.patch ]]
          + chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
          + /data/hive-ptest/working/scratch/smart-apply-patch.sh /data/hive-ptest/working/scratch/build.patch
          The patch does not appear to apply with p0 to p2
          + exit 1
          '
          

          This message is automatically generated.

          Show
          Hive QA added a comment - Overall : -1 no tests executed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12594435/HIVE-4827.4.patch Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/199/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/199/console Messages: Executing org.apache.hive.ptest.execution.CleanupPhase Executing org.apache.hive.ptest.execution.PrepPhase Tests failed with: NonZeroExitCodeException: Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ [[ -n '' ]] + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128' + cd /data/hive-ptest/working/ + tee /data/hive-ptest/logs/PreCommit-HIVE-Build-199/source-prep.txt + mkdir -p maven ivy + [[ svn = \s\v\n ]] + [[ -n '' ]] + [[ -d apache-svn-trunk-source ]] + [[ ! -d apache-svn-trunk-source/.svn ]] + [[ ! -d apache-svn-trunk-source ]] + cd apache-svn-trunk-source + svn revert -R . Reverted 'contrib/src/test/results/clientpositive/serde_typedbytes4.q.out' Reverted 'contrib/src/test/results/clientpositive/dboutput.q.out' Reverted 'ql/src/test/results/clientpositive/auto_join27.q.out' Reverted 'ql/src/test/results/clientpositive/join14_hadoop20.q.out' Reverted 'ql/src/test/results/clientpositive/udf9.q.out' Reverted 'ql/src/test/results/clientpositive/ppd_clusterby.q.out' Reverted 'ql/src/test/results/clientpositive/auto_join13.q.out' Reverted 'ql/src/test/results/clientpositive/groupby_multi_single_reducer2.q.out' Reverted 'ql/src/test/results/clientpositive/index_auto_update.q.out' Reverted 'ql/src/test/results/clientpositive/notable_alias1.q.out' Reverted 'ql/src/test/results/clientpositive/load_dyn_part13.q.out' Reverted 'ql/src/test/results/clientpositive/udf_union.q.out' Reverted 'ql/src/test/results/clientpositive/sort_merge_join_desc_3.q.out' Reverted 'ql/src/test/results/clientpositive/join38.q.out' Reverted 'ql/src/test/results/clientpositive/input_part7.q.out' Reverted 'ql/src/test/results/clientpositive/groupby_grouping_sets4.q.out' Reverted 'ql/src/test/results/clientpositive/pcr.q.out' Reverted 'ql/src/test/results/clientpositive/mapjoin_filter_on_outerjoin.q.out' Reverted 'ql/src/test/results/clientpositive/index_auto_partitioned.q.out' Reverted 'ql/src/test/results/clientpositive/cluster.q.out' Reverted 'ql/src/test/results/clientpositive/input12_hadoop20.q.out' Reverted 'ql/src/test/results/clientpositive/index_auto_file_format.q.out' Reverted 'ql/src/test/results/clientpositive/input_part2.q.out' Reverted 'ql/src/test/results/clientpositive/ppd_transform.q.out' Reverted 'ql/src/test/results/clientpositive/create_view.q.out' Reverted 'ql/src/test/results/clientpositive/alter_partition_coltype.q.out' Reverted 'ql/src/test/results/clientpositive/infer_const_type.q.out' Reverted 'ql/src/test/results/clientpositive/correlationoptimizer8.q.out' Reverted 'ql/src/test/results/clientpositive/join7.q.out' Reverted 'ql/src/test/results/clientpositive/auto_join4.q.out' Reverted 'ql/src/test/results/clientpositive/input2_limit.q.out' Reverted 'ql/src/test/results/clientpositive/input11_limit.q.out' Reverted 'ql/src/test/results/clientpositive/udf_like.q.out' Reverted 'ql/src/test/results/clientpositive/udf_hour.q.out' Reverted 'ql/src/test/results/clientpositive/groupby_sort_1.q.out' Reverted 'ql/src/test/results/clientpositive/ppr_pushdown3.q.out' Reverted 'ql/src/test/results/clientpositive/input11.q.out' Reverted 'ql/src/test/results/clientpositive/noalias_subq1.q.out' Reverted 'ql/src/test/results/clientpositive/auto_join29.q.out' Reverted 'ql/src/test/results/clientpositive/index_auto_mult_tables_compact.q.out' Reverted 'ql/src/test/results/clientpositive/regex_col.q.out' Reverted 'ql/src/test/results/clientpositive/allcolref_in_udf.q.out' Reverted 'ql/src/test/results/clientpositive/udf1.q.out' Reverted 'ql/src/test/results/clientpositive/join12.q.out' Reverted 'ql/src/test/results/clientpositive/join35.q.out' Reverted 'ql/src/test/results/clientpositive/input_part4.q.out' Reverted 'ql/src/test/results/clientpositive/union20.q.out' Reverted 'ql/src/test/results/clientpositive/column_access_stats.q.out' Reverted 'ql/src/test/results/clientpositive/join21.q.out' Reverted 'ql/src/test/results/clientpositive/ppd_multi_insert.q.out' Reverted 'ql/src/test/results/clientpositive/index_stale_partitioned.q.out' Reverted 'ql/src/test/results/clientpositive/auto_join6.q.out' Reverted 'ql/src/test/results/clientpositive/correlationoptimizer10.q.out' Reverted 'ql/src/test/results/clientpositive/udf_lower.q.out' Reverted 'ql/src/test/results/clientpositive/nullgroup.q.out' Reverted 'ql/src/test/results/clientpositive/join4.q.out' Reverted 'ql/src/test/results/clientpositive/nullgroup4_multi_distinct.q.out' Reverted 'ql/src/test/results/clientpositive/index_bitmap_compression.q.out' Reverted 'ql/src/test/results/clientpositive/input18.q.out' Reverted 'ql/src/test/results/clientpositive/multi_insert_gby.q.out' Reverted 'ql/src/test/results/clientpositive/groupby_multi_single_reducer.q.out' Reverted 'ql/src/test/results/clientpositive/index_auto_mult_tables.q.out' Reverted 'ql/src/test/results/clientpositive/rand_partitionpruner3.q.out' Reverted 'ql/src/test/results/clientpositive/input13.q.out' Reverted 'ql/src/test/results/clientpositive/order2.q.out' Reverted 'ql/src/test/results/clientpositive/multi_insert.q.out' Reverted 'ql/src/test/results/clientpositive/ppd_udf_col.q.out' Reverted 'ql/src/test/results/clientpositive/auto_join12.q.out' Reverted 'ql/src/test/results/clientpositive/ppd_outer_join1.q.out' Reverted 'ql/src/test/results/clientpositive/ppd2.q.out' Reverted 'ql/src/test/results/clientpositive/index_auto_empty.q.out' Reverted 'ql/src/test/results/clientpositive/auto_join21.q.out' Reverted 'ql/src/test/results/clientpositive/sort_merge_join_desc_2.q.out' Reverted 'ql/src/test/results/clientpositive/union22.q.out' Reverted 'ql/src/test/results/clientpositive/join23.q.out' Reverted 'ql/src/test/results/clientpositive/udf_second.q.out' Reverted 'ql/src/test/results/clientpositive/index_auto_multiple.q.out' Reverted 'ql/src/test/results/clientpositive/union.q.out' Reverted 'ql/src/test/results/clientpositive/udf_10_trims.q.out' Reverted 'ql/src/test/results/clientpositive/multi_insert_move_tasks_share_dependencies.q.out' Reverted 'ql/src/test/results/clientpositive/input_part1.q.out' Reverted 'ql/src/test/results/clientpositive/groupby_sort_skew_1.q.out' Reverted 'ql/src/test/results/clientpositive/auto_join8.q.out' Reverted 'ql/src/test/results/clientpositive/index_compression.q.out' Reverted 'ql/src/test/results/clientpositive/udf_parse_url.q.out' Reverted 'ql/src/test/results/clientpositive/join41.q.out' Reverted 'ql/src/test/results/clientpositive/multi_insert_lateral_view.q.out' Reverted 'ql/src/test/results/clientpositive/no_hooks.q.out' Reverted 'ql/src/test/results/clientpositive/join6.q.out' Reverted 'ql/src/test/results/clientpositive/index_bitmap3.q.out' Reverted 'ql/src/test/results/clientpositive/groupby_position.q.out' Reverted 'ql/src/test/results/clientpositive/transform_ppr2.q.out' Reverted 'ql/src/test/results/clientpositive/set_processor_namespaces.q.out' Reverted 'ql/src/test/results/clientpositive/union_ppr.q.out' Reverted 'ql/src/test/results/clientpositive/auto_join14_hadoop20.q.out' Reverted 'ql/src/test/results/clientpositive/index_bitmap_auto_partitioned.q.out' Reverted 'ql/src/test/results/clientpositive/nonmr_fetch.q.out' Reverted 'ql/src/test/results/clientpositive/router_join_ppr.q.out' Reverted 'ql/src/test/results/clientpositive/auto_join28.q.out' Reverted 'ql/src/test/results/clientpositive/groupby_multi_single_reducer3.q.out' Reverted 'ql/src/test/results/clientpositive/udf_minute.q.out' Reverted 'ql/src/test/results/clientpositive/input42.q.out' Reverted 'ql/src/test/results/clientpositive/having.q.out' Reverted 'ql/src/test/results/clientpositive/quote1.q.out' Reverted 'ql/src/test/results/clientpositive/notable_alias2.q.out' Reverted 'ql/src/test/results/clientpositive/index_auto_unused.q.out' Reverted 'ql/src/test/results/clientpositive/auto_join23.q.out' Reverted 'ql/src/test/results/clientpositive/join16.q.out' Reverted 'ql/src/test/results/clientpositive/sort_merge_join_desc_4.q.out' Reverted 'ql/src/test/results/clientpositive/nullgroup2.q.out' Reverted 'ql/src/test/results/clientpositive/join39.q.out' Reverted 'ql/src/test/results/clientpositive/union24.q.out' Reverted 'ql/src/test/results/clientpositive/index_auto.q.out' Reverted 'ql/src/test/results/clientpositive/rcfile_null_value.q.out' Reverted 'ql/src/test/results/clientpositive/union33.q.out' Reverted 'ql/src/test/results/clientpositive/join11.q.out' Reverted 'ql/src/test/results/clientpositive/join34.q.out' Reverted 'ql/src/test/results/clientpositive/join20.q.out' Reverted 'ql/src/test/results/clientpositive/join8.q.out' Reverted 'ql/src/test/results/clientpositive/subq.q.out' Reverted 'ql/src/test/results/clientpositive/auto_join5.q.out' Reverted 'ql/src/test/results/clientpositive/input14_limit.q.out' Reverted 'ql/src/test/results/clientpositive/louter_join_ppr.q.out' Reverted 'ql/src/test/results/clientpositive/auto_join0.q.out' Reverted 'ql/src/test/results/clientpositive/skewjoin.q.out' Reverted 'ql/src/test/results/clientpositive/index_auto_self_join.q.out' Reverted 'ql/src/test/results/clientpositive/auto_join16.q.out' Reverted 'ql/src/test/results/clientpositive/nullgroup4.q.out' Reverted 'ql/src/test/results/clientpositive/auto_join11.q.out' Reverted 'ql/src/test/results/clientpositive/subq2.q.out' Reverted 'ql/src/test/results/clientpositive/auto_join20.q.out' Reverted 'ql/src/test/results/clientpositive/sort_merge_join_desc_1.q.out' Reverted 'ql/src/test/results/clientpositive/join13.q.out' Reverted 'ql/src/test/results/clientpositive/input_part5.q.out' Reverted 'ql/src/test/results/clientpositive/cast1.q.out' Reverted 'ql/src/test/results/clientpositive/index_stale.q.out' Reverted 'ql/src/test/results/clientpositive/auto_join7.q.out' Reverted 'ql/src/test/results/clientpositive/join40.q.out' Reverted 'ql/src/test/results/clientpositive/join5.q.out' Reverted 'ql/src/test/results/clientpositive/outer_join_ppr.q.out' Reverted 'ql/src/test/results/clientpositive/input1_limit.q.out' Reverted 'ql/src/test/results/clientpositive/index_bitmap_auto.q.out' Reverted 'ql/src/test/results/clientpositive/transform_ppr1.q.out' Reverted 'ql/src/test/results/clientpositive/join0.q.out' Reverted 'ql/src/test/results/clientpositive/regexp_extract.q.out' Reverted 'ql/src/test/results/clientpositive/ctas_colname.q.out' Reverted 'ql/src/test/results/clientpositive/mergejoins.q.out' Reverted 'ql/src/test/results/clientpositive/input14.q.out' Reverted 'ql/src/test/results/beelinepositive/auto_join27.q.out' Reverted 'ql/src/test/results/beelinepositive/auto_join13.q.out' Reverted 'ql/src/test/results/beelinepositive/create_view.q.out' Reverted 'ql/src/test/results/beelinepositive/auto_join28.q.out' Reverted 'ql/src/test/results/beelinepositive/auto_join4.q.out' Reverted 'ql/src/test/results/beelinepositive/auto_join23.q.out' Reverted 'ql/src/test/results/beelinepositive/auto_join29.q.out' Reverted 'ql/src/test/results/beelinepositive/auto_join5.q.out' Reverted 'ql/src/test/results/beelinepositive/auto_join0.q.out' Reverted 'ql/src/test/results/beelinepositive/auto_join6.q.out' Reverted 'ql/src/test/results/beelinepositive/auto_join16.q.out' Reverted 'ql/src/test/results/beelinepositive/auto_join20.q.out' Reverted 'ql/src/test/results/beelinepositive/auto_join7.q.out' Reverted 'ql/src/test/results/beelinepositive/auto_join12.q.out' Reverted 'ql/src/test/results/beelinepositive/auto_join21.q.out' Reverted 'ql/src/test/results/beelinepositive/auto_join8.q.out' Reverted 'ql/src/test/results/compiler/plan/input2.q.xml' Reverted 'ql/src/test/results/compiler/plan/input3.q.xml' Reverted 'ql/src/test/results/compiler/plan/join4.q.xml' Reverted 'ql/src/test/results/compiler/plan/input4.q.xml' Reverted 'ql/src/test/results/compiler/plan/join5.q.xml' Reverted 'ql/src/test/results/compiler/plan/join6.q.xml' Reverted 'ql/src/test/results/compiler/plan/join7.q.xml' Reverted 'ql/src/test/results/compiler/plan/join8.q.xml' Reverted 'ql/src/test/results/compiler/plan/union.q.xml' Reverted 'ql/src/test/results/compiler/plan/udf1.q.xml' Reverted 'ql/src/test/results/compiler/plan/input_part1.q.xml' Reverted 'ql/src/test/results/compiler/plan/subq.q.xml' Reverted 'ql/src/test/results/compiler/plan/cast1.q.xml' Reverted 'ql/src/test/results/compiler/plan/input1.q.xml' Reverted 'ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java' ++ egrep -v '^X|^Performing status on external' ++ awk '{print $2}' ++ svn status --no-ignore + rm -rf build hcatalog/build hcatalog/core/build hcatalog/storage-handlers/hbase/build hcatalog/server-extensions/build hcatalog/webhcat/svr/build hcatalog/webhcat/java-client/build hcatalog/hcatalog-pig-adapter/build common/src/gen + svn update D testutils/ptest2/src/test/resources/TEST-SomeTest-failure.xml D testutils/ptest2/src/test/resources/test-outputs/TEST-skewjoin_union_remove_1.q-6fa31776-d2b0-4e13-9761-11f750627ad1-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml D testutils/ptest2/src/test/resources/test-outputs/TEST-union_remove_9.q-acb9de8f-1b9c-4874-924c-b2107ca7b07c-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml D testutils/ptest2/src/test/resources/test-outputs/TEST-skewjoin.q-ab8536a7-1b5c-45ed-ba29-14450f27db8b-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml D testutils/ptest2/src/test/resources/test-outputs/TEST-index_auth.q-bucketcontex-ba31fb54-1d7f-4c70-a89d-477b7d155191-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml D testutils/ptest2/src/test/resources/test-outputs/TEST-SomeTest-truncated.xml Fetching external item into 'hcatalog/src/test/e2e/harness' Updated external to revision 1507508. Updated to revision 1507508. + patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hive-ptest/working/scratch/build.patch + [[ -f /data/hive-ptest/working/scratch/build.patch ]] + chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh + /data/hive-ptest/working/scratch/smart-apply-patch.sh /data/hive-ptest/working/scratch/build.patch The patch does not appear to apply with p0 to p2 + exit 1 ' This message is automatically generated.
          Yin Huai made changes -
          Attachment HIVE-4827.5.patch [ 12594491 ]
          Hide
          Hive QA added a comment -

          Overall: +1 all checks pass

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12594491/HIVE-4827.5.patch

          SUCCESS: +1 2653 tests passed

          Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/202/testReport
          Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/202/console

          Messages:

          Executing org.apache.hive.ptest.execution.CleanupPhase
          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          

          This message is automatically generated.

          Show
          Hive QA added a comment - Overall : +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12594491/HIVE-4827.5.patch SUCCESS: +1 2653 tests passed Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/202/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/202/console Messages: Executing org.apache.hive.ptest.execution.CleanupPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase This message is automatically generated.
          Hide
          Yin Huai added a comment -

          update

          Show
          Yin Huai added a comment - update
          Yin Huai made changes -
          Attachment HIVE-4827.6.patch [ 12595028 ]
          Hide
          Gunther Hagleitner added a comment -

          This looks really good. Just a few smaller things:

          • Can you add tests for cascading mapjoins? Something like 5 joins that will produce two groups 3 + 2.
          • Can you add test to show that setting the threshold to 0 effectively turns off the optimization?

          Finally, I think it makes sense to remove the old flag since it no longer applies. But can you fill out the release notes section of this ticket and describe how 'optimize.mapjoin.mapreduce' is superseded by threshold = 0 or noconditionaltask?

          Show
          Gunther Hagleitner added a comment - This looks really good. Just a few smaller things: Can you add tests for cascading mapjoins? Something like 5 joins that will produce two groups 3 + 2. Can you add test to show that setting the threshold to 0 effectively turns off the optimization? Finally, I think it makes sense to remove the old flag since it no longer applies. But can you fill out the release notes section of this ticket and describe how 'optimize.mapjoin.mapreduce' is superseded by threshold = 0 or noconditionaltask?
          Hide
          Yin Huai added a comment -

          canceling patch. Will update later

          Show
          Yin Huai added a comment - canceling patch. Will update later
          Yin Huai made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Yin Huai made changes -
          Release Note Before applying this jira to trunk, CommonJoinTaskDispatcher has two methods, mergeMapJoinTaskWithChildMapJoinTask and mergeMapJoinTaskWithMapReduceTask. The first method tries to merge a map-only task (for MapJoin) to its child map-only task. The second method tries to merge a map-only task to its child MapReduce task (a task has a reducer). There was a flag called "hive.optimize.mapjoin.mapreduce" to determine if mergeMapJoinTaskWithMapReduceTask will be called.

          This work combines mergeMapJoinTaskWithChildMapJoinTask and mergeMapJoinTaskWithMapReduceTask. So, a map-only task will be merged into its child task no matter the child task is a map-only task or a MapReduce task. So hive.optimize.mapjoin.mapreduce is not needed any more.

          If a user wants to disable merging a map-only task to its child task, he or she can use either 'set hive.auto.convert.join.noconditionaltask=false' or 'set hive.auto.convert.join.noconditionaltask=true;
          set hive.auto.convert.join.noconditionaltask.size=0;'
          Yin Huai made changes -
          Release Note Before applying this jira to trunk, CommonJoinTaskDispatcher has two methods, mergeMapJoinTaskWithChildMapJoinTask and mergeMapJoinTaskWithMapReduceTask. The first method tries to merge a map-only task (for MapJoin) to its child map-only task. The second method tries to merge a map-only task to its child MapReduce task (a task has a reducer). There was a flag called "hive.optimize.mapjoin.mapreduce" to determine if mergeMapJoinTaskWithMapReduceTask will be called.

          This work combines mergeMapJoinTaskWithChildMapJoinTask and mergeMapJoinTaskWithMapReduceTask. So, a map-only task will be merged into its child task no matter the child task is a map-only task or a MapReduce task. So hive.optimize.mapjoin.mapreduce is not needed any more.

          If a user wants to disable merging a map-only task to its child task, he or she can use either 'set hive.auto.convert.join.noconditionaltask=false' or 'set hive.auto.convert.join.noconditionaltask=true;
          set hive.auto.convert.join.noconditionaltask.size=0;'
          Before applying this jira to trunk, CommonJoinTaskDispatcher has two methods, mergeMapJoinTaskWithChildMapJoinTask and mergeMapJoinTaskWithMapReduceTask. The first method tries to merge a map-only task (for MapJoin) to its child map-only task. The second method tries to merge a map-only task to its child MapReduce task (a task has a reducer). There was a flag called "hive.optimize.mapjoin.mapreduce" to determine if mergeMapJoinTaskWithMapReduceTask will be called.

          This work combines mergeMapJoinTaskWithChildMapJoinTask and mergeMapJoinTaskWithMapReduceTask. So, a map-only task will be merged into its child task no matter the child task is a map-only task or a MapReduce task. So hive.optimize.mapjoin.mapreduce is not needed any more.

          If a user wants to disable merging a map-only task to its child task, he or she can use either
          {code}
          set hive.auto.convert.join.noconditionaltask=false;
          {\code}

          {code}
          set hive.auto.convert.join.noconditionaltask=true;
          set hive.auto.convert.join.noconditionaltask.size=0;
          {\code}
          Yin Huai made changes -
          Release Note Before applying this jira to trunk, CommonJoinTaskDispatcher has two methods, mergeMapJoinTaskWithChildMapJoinTask and mergeMapJoinTaskWithMapReduceTask. The first method tries to merge a map-only task (for MapJoin) to its child map-only task. The second method tries to merge a map-only task to its child MapReduce task (a task has a reducer). There was a flag called "hive.optimize.mapjoin.mapreduce" to determine if mergeMapJoinTaskWithMapReduceTask will be called.

          This work combines mergeMapJoinTaskWithChildMapJoinTask and mergeMapJoinTaskWithMapReduceTask. So, a map-only task will be merged into its child task no matter the child task is a map-only task or a MapReduce task. So hive.optimize.mapjoin.mapreduce is not needed any more.

          If a user wants to disable merging a map-only task to its child task, he or she can use either
          {code}
          set hive.auto.convert.join.noconditionaltask=false;
          {\code}

          {code}
          set hive.auto.convert.join.noconditionaltask=true;
          set hive.auto.convert.join.noconditionaltask.size=0;
          {\code}
          Before applying this jira to trunk, CommonJoinTaskDispatcher has two methods, mergeMapJoinTaskWithChildMapJoinTask and mergeMapJoinTaskWithMapReduceTask. The first method tries to merge a map-only task (for MapJoin) to its child map-only task. The second method tries to merge a map-only task to its child MapReduce task (a task has a reducer). There was a flag called "hive.optimize.mapjoin.mapreduce" to determine if mergeMapJoinTaskWithMapReduceTask will be called.

          This work combines mergeMapJoinTaskWithChildMapJoinTask and mergeMapJoinTaskWithMapReduceTask. So, a map-only task will be merged into its child task no matter the child task is a map-only task or a MapReduce task. So hive.optimize.mapjoin.mapreduce is not needed any more.

          If a user wants to disable merging a map-only task to its child task, he or she can use either
          set hive.auto.convert.join.noconditionaltask=false;
          or
          set hive.auto.convert.join.noconditionaltask=true;
          set hive.auto.convert.join.noconditionaltask.size=0;
          Hide
          Yin Huai added a comment -

          addressed Gunther's comments

          Show
          Yin Huai added a comment - addressed Gunther's comments
          Yin Huai made changes -
          Attachment HIVE-4827.7.patch [ 12595092 ]
          Yin Huai made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Yin Huai made changes -
          Summary Merge a Map-only job to its following MapReduce job with multiple inputs Merge a Map-only task to its child task
          Hide
          Gunther Hagleitner added a comment -

          This looks good to me. Thanks for adding the tests that show how joins are split on memory boundaries. The code now does more and is simpler IMO. Nice!

          +1

          Will commit if tests pass.

          Show
          Gunther Hagleitner added a comment - This looks good to me. Thanks for adding the tests that show how joins are split on memory boundaries. The code now does more and is simpler IMO. Nice! +1 Will commit if tests pass.
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12595092/HIVE-4827.7.patch

          ERROR: -1 due to 1 failed/errored test(s), 2748 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_12
          

          Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/253/testReport
          Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/253/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests failed with: TestsFailedException: 1 tests failed
          

          This message is automatically generated.

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12595092/HIVE-4827.7.patch ERROR: -1 due to 1 failed/errored test(s), 2748 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_12 Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/253/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/253/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests failed with: TestsFailedException: 1 tests failed This message is automatically generated.
          Yin Huai made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Yin Huai added a comment -

          my bad... the golden file of auto_sortmerge_join_12.q has not been updated after I rebased the code...

          Show
          Yin Huai added a comment - my bad... the golden file of auto_sortmerge_join_12.q has not been updated after I rebased the code...
          Hide
          Yin Huai added a comment -

          update the golden file of auto_sortmerge_join_12.q

          Show
          Yin Huai added a comment - update the golden file of auto_sortmerge_join_12.q
          Yin Huai made changes -
          Attachment HIVE-4827.8.patch [ 12595120 ]
          Yin Huai made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hive QA added a comment -

          Overall: +1 all checks pass

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12595120/HIVE-4827.8.patch

          SUCCESS: +1 2748 tests passed

          Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/258/testReport
          Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/258/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          

          This message is automatically generated.

          Show
          Hive QA added a comment - Overall : +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12595120/HIVE-4827.8.patch SUCCESS: +1 2748 tests passed Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/258/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/258/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase This message is automatically generated.
          Hide
          Gunther Hagleitner added a comment -

          Committed to trunk. Thanks Yin!

          Show
          Gunther Hagleitner added a comment - Committed to trunk. Thanks Yin!
          Gunther Hagleitner made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Ashutosh Chauhan made changes -
          Fix Version/s 0.12.0 [ 12324312 ]
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hive-trunk-hadoop2-ptest #39 (See https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/39/)
          HIVE-4827: Merge a Map-only task to its child task (Yin Huai via Gunther Hagleitner) (gunther: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1509082)

          • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          • /hive/trunk/conf/hive-default.xml.template
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorUtils.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/correlation/QueryPlanTreeTransformation.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/CommonJoinTaskDispatcher.java
          • /hive/trunk/ql/src/test/queries/clientpositive/auto_join33.q
          • /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer1.q
          • /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer3.q
          • /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer4.q
          • /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer5.q
          • /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer6.q
          • /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer7.q
          • /hive/trunk/ql/src/test/queries/clientpositive/multiMapJoin1.q
          • /hive/trunk/ql/src/test/queries/clientpositive/multiMapJoin2.q
          • /hive/trunk/ql/src/test/queries/clientpositive/union34.q
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join0.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join10.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join11.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join12.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join13.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join15.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join16.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join20.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join21.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join22.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join23.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join24.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join26.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join28.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join29.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join32.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join33.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_10.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_11.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_12.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_9.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer1.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer3.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer4.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer6.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer7.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/join28.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/join32.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/join33.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/join_star.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/mapjoin_filter_on_outerjoin.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/mapjoin_mapjoin.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/mapjoin_subquery.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/mapjoin_subquery2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/mapjoin_test_outer.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/multiMapJoin1.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/multiMapJoin2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/multi_join_union.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/union34.q.out
          Show
          Hudson added a comment - FAILURE: Integrated in Hive-trunk-hadoop2-ptest #39 (See https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/39/ ) HIVE-4827 : Merge a Map-only task to its child task (Yin Huai via Gunther Hagleitner) (gunther: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1509082 ) /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/conf/hive-default.xml.template /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorUtils.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/correlation/QueryPlanTreeTransformation.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/CommonJoinTaskDispatcher.java /hive/trunk/ql/src/test/queries/clientpositive/auto_join33.q /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer1.q /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer3.q /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer4.q /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer5.q /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer6.q /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer7.q /hive/trunk/ql/src/test/queries/clientpositive/multiMapJoin1.q /hive/trunk/ql/src/test/queries/clientpositive/multiMapJoin2.q /hive/trunk/ql/src/test/queries/clientpositive/union34.q /hive/trunk/ql/src/test/results/clientpositive/auto_join0.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join10.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join11.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join12.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join13.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join15.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join16.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join2.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join20.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join21.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join22.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join23.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join24.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join26.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join28.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join29.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join32.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join33.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_10.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_11.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_12.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_9.q.out /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer1.q.out /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer3.q.out /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer4.q.out /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer6.q.out /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer7.q.out /hive/trunk/ql/src/test/results/clientpositive/join28.q.out /hive/trunk/ql/src/test/results/clientpositive/join32.q.out /hive/trunk/ql/src/test/results/clientpositive/join33.q.out /hive/trunk/ql/src/test/results/clientpositive/join_star.q.out /hive/trunk/ql/src/test/results/clientpositive/mapjoin_filter_on_outerjoin.q.out /hive/trunk/ql/src/test/results/clientpositive/mapjoin_mapjoin.q.out /hive/trunk/ql/src/test/results/clientpositive/mapjoin_subquery.q.out /hive/trunk/ql/src/test/results/clientpositive/mapjoin_subquery2.q.out /hive/trunk/ql/src/test/results/clientpositive/mapjoin_test_outer.q.out /hive/trunk/ql/src/test/results/clientpositive/multiMapJoin1.q.out /hive/trunk/ql/src/test/results/clientpositive/multiMapJoin2.q.out /hive/trunk/ql/src/test/results/clientpositive/multi_join_union.q.out /hive/trunk/ql/src/test/results/clientpositive/union34.q.out
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hive-trunk-hadoop1-ptest #111 (See https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/111/)
          HIVE-4827: Merge a Map-only task to its child task (Yin Huai via Gunther Hagleitner) (gunther: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1509082)

          • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          • /hive/trunk/conf/hive-default.xml.template
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorUtils.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/correlation/QueryPlanTreeTransformation.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/CommonJoinTaskDispatcher.java
          • /hive/trunk/ql/src/test/queries/clientpositive/auto_join33.q
          • /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer1.q
          • /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer3.q
          • /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer4.q
          • /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer5.q
          • /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer6.q
          • /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer7.q
          • /hive/trunk/ql/src/test/queries/clientpositive/multiMapJoin1.q
          • /hive/trunk/ql/src/test/queries/clientpositive/multiMapJoin2.q
          • /hive/trunk/ql/src/test/queries/clientpositive/union34.q
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join0.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join10.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join11.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join12.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join13.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join15.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join16.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join20.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join21.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join22.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join23.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join24.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join26.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join28.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join29.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join32.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join33.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_10.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_11.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_12.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_9.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer1.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer3.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer4.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer6.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer7.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/join28.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/join32.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/join33.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/join_star.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/mapjoin_filter_on_outerjoin.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/mapjoin_mapjoin.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/mapjoin_subquery.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/mapjoin_subquery2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/mapjoin_test_outer.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/multiMapJoin1.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/multiMapJoin2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/multi_join_union.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/union34.q.out
          Show
          Hudson added a comment - FAILURE: Integrated in Hive-trunk-hadoop1-ptest #111 (See https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/111/ ) HIVE-4827 : Merge a Map-only task to its child task (Yin Huai via Gunther Hagleitner) (gunther: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1509082 ) /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/conf/hive-default.xml.template /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorUtils.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/correlation/QueryPlanTreeTransformation.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/CommonJoinTaskDispatcher.java /hive/trunk/ql/src/test/queries/clientpositive/auto_join33.q /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer1.q /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer3.q /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer4.q /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer5.q /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer6.q /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer7.q /hive/trunk/ql/src/test/queries/clientpositive/multiMapJoin1.q /hive/trunk/ql/src/test/queries/clientpositive/multiMapJoin2.q /hive/trunk/ql/src/test/queries/clientpositive/union34.q /hive/trunk/ql/src/test/results/clientpositive/auto_join0.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join10.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join11.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join12.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join13.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join15.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join16.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join2.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join20.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join21.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join22.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join23.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join24.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join26.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join28.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join29.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join32.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join33.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_10.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_11.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_12.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_9.q.out /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer1.q.out /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer3.q.out /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer4.q.out /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer6.q.out /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer7.q.out /hive/trunk/ql/src/test/results/clientpositive/join28.q.out /hive/trunk/ql/src/test/results/clientpositive/join32.q.out /hive/trunk/ql/src/test/results/clientpositive/join33.q.out /hive/trunk/ql/src/test/results/clientpositive/join_star.q.out /hive/trunk/ql/src/test/results/clientpositive/mapjoin_filter_on_outerjoin.q.out /hive/trunk/ql/src/test/results/clientpositive/mapjoin_mapjoin.q.out /hive/trunk/ql/src/test/results/clientpositive/mapjoin_subquery.q.out /hive/trunk/ql/src/test/results/clientpositive/mapjoin_subquery2.q.out /hive/trunk/ql/src/test/results/clientpositive/mapjoin_test_outer.q.out /hive/trunk/ql/src/test/results/clientpositive/multiMapJoin1.q.out /hive/trunk/ql/src/test/results/clientpositive/multiMapJoin2.q.out /hive/trunk/ql/src/test/results/clientpositive/multi_join_union.q.out /hive/trunk/ql/src/test/results/clientpositive/union34.q.out
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hive-trunk-h0.21 #2237 (See https://builds.apache.org/job/Hive-trunk-h0.21/2237/)
          HIVE-4827: Merge a Map-only task to its child task (Yin Huai via Gunther Hagleitner) (gunther: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1509082)

          • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          • /hive/trunk/conf/hive-default.xml.template
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorUtils.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/correlation/QueryPlanTreeTransformation.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/CommonJoinTaskDispatcher.java
          • /hive/trunk/ql/src/test/queries/clientpositive/auto_join33.q
          • /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer1.q
          • /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer3.q
          • /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer4.q
          • /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer5.q
          • /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer6.q
          • /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer7.q
          • /hive/trunk/ql/src/test/queries/clientpositive/multiMapJoin1.q
          • /hive/trunk/ql/src/test/queries/clientpositive/multiMapJoin2.q
          • /hive/trunk/ql/src/test/queries/clientpositive/union34.q
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join0.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join10.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join11.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join12.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join13.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join15.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join16.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join20.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join21.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join22.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join23.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join24.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join26.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join28.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join29.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join32.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join33.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_10.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_11.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_12.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_9.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer1.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer3.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer4.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer6.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer7.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/join28.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/join32.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/join33.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/join_star.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/mapjoin_filter_on_outerjoin.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/mapjoin_mapjoin.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/mapjoin_subquery.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/mapjoin_subquery2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/mapjoin_test_outer.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/multiMapJoin1.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/multiMapJoin2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/multi_join_union.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/union34.q.out
          Show
          Hudson added a comment - FAILURE: Integrated in Hive-trunk-h0.21 #2237 (See https://builds.apache.org/job/Hive-trunk-h0.21/2237/ ) HIVE-4827 : Merge a Map-only task to its child task (Yin Huai via Gunther Hagleitner) (gunther: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1509082 ) /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/conf/hive-default.xml.template /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorUtils.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/correlation/QueryPlanTreeTransformation.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/CommonJoinTaskDispatcher.java /hive/trunk/ql/src/test/queries/clientpositive/auto_join33.q /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer1.q /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer3.q /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer4.q /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer5.q /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer6.q /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer7.q /hive/trunk/ql/src/test/queries/clientpositive/multiMapJoin1.q /hive/trunk/ql/src/test/queries/clientpositive/multiMapJoin2.q /hive/trunk/ql/src/test/queries/clientpositive/union34.q /hive/trunk/ql/src/test/results/clientpositive/auto_join0.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join10.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join11.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join12.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join13.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join15.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join16.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join2.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join20.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join21.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join22.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join23.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join24.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join26.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join28.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join29.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join32.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join33.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_10.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_11.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_12.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_9.q.out /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer1.q.out /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer3.q.out /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer4.q.out /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer6.q.out /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer7.q.out /hive/trunk/ql/src/test/results/clientpositive/join28.q.out /hive/trunk/ql/src/test/results/clientpositive/join32.q.out /hive/trunk/ql/src/test/results/clientpositive/join33.q.out /hive/trunk/ql/src/test/results/clientpositive/join_star.q.out /hive/trunk/ql/src/test/results/clientpositive/mapjoin_filter_on_outerjoin.q.out /hive/trunk/ql/src/test/results/clientpositive/mapjoin_mapjoin.q.out /hive/trunk/ql/src/test/results/clientpositive/mapjoin_subquery.q.out /hive/trunk/ql/src/test/results/clientpositive/mapjoin_subquery2.q.out /hive/trunk/ql/src/test/results/clientpositive/mapjoin_test_outer.q.out /hive/trunk/ql/src/test/results/clientpositive/multiMapJoin1.q.out /hive/trunk/ql/src/test/results/clientpositive/multiMapJoin2.q.out /hive/trunk/ql/src/test/results/clientpositive/multi_join_union.q.out /hive/trunk/ql/src/test/results/clientpositive/union34.q.out
          Hide
          Hudson added a comment -

          ABORTED: Integrated in Hive-trunk-hadoop2 #320 (See https://builds.apache.org/job/Hive-trunk-hadoop2/320/)
          HIVE-4827: Merge a Map-only task to its child task (Yin Huai via Gunther Hagleitner) (gunther: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1509082)

          • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          • /hive/trunk/conf/hive-default.xml.template
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorUtils.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/correlation/QueryPlanTreeTransformation.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/CommonJoinTaskDispatcher.java
          • /hive/trunk/ql/src/test/queries/clientpositive/auto_join33.q
          • /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer1.q
          • /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer3.q
          • /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer4.q
          • /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer5.q
          • /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer6.q
          • /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer7.q
          • /hive/trunk/ql/src/test/queries/clientpositive/multiMapJoin1.q
          • /hive/trunk/ql/src/test/queries/clientpositive/multiMapJoin2.q
          • /hive/trunk/ql/src/test/queries/clientpositive/union34.q
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join0.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join10.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join11.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join12.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join13.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join15.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join16.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join20.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join21.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join22.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join23.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join24.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join26.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join28.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join29.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join32.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_join33.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_10.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_11.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_12.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_9.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer1.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer3.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer4.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer6.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer7.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/join28.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/join32.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/join33.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/join_star.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/mapjoin_filter_on_outerjoin.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/mapjoin_mapjoin.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/mapjoin_subquery.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/mapjoin_subquery2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/mapjoin_test_outer.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/multiMapJoin1.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/multiMapJoin2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/multi_join_union.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/union34.q.out
          Show
          Hudson added a comment - ABORTED: Integrated in Hive-trunk-hadoop2 #320 (See https://builds.apache.org/job/Hive-trunk-hadoop2/320/ ) HIVE-4827 : Merge a Map-only task to its child task (Yin Huai via Gunther Hagleitner) (gunther: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1509082 ) /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/conf/hive-default.xml.template /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorUtils.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/correlation/QueryPlanTreeTransformation.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/CommonJoinTaskDispatcher.java /hive/trunk/ql/src/test/queries/clientpositive/auto_join33.q /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer1.q /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer3.q /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer4.q /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer5.q /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer6.q /hive/trunk/ql/src/test/queries/clientpositive/correlationoptimizer7.q /hive/trunk/ql/src/test/queries/clientpositive/multiMapJoin1.q /hive/trunk/ql/src/test/queries/clientpositive/multiMapJoin2.q /hive/trunk/ql/src/test/queries/clientpositive/union34.q /hive/trunk/ql/src/test/results/clientpositive/auto_join0.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join10.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join11.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join12.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join13.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join15.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join16.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join2.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join20.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join21.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join22.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join23.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join24.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join26.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join28.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join29.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join32.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_join33.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_10.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_11.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_12.q.out /hive/trunk/ql/src/test/results/clientpositive/auto_sortmerge_join_9.q.out /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer1.q.out /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer3.q.out /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer4.q.out /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer6.q.out /hive/trunk/ql/src/test/results/clientpositive/correlationoptimizer7.q.out /hive/trunk/ql/src/test/results/clientpositive/join28.q.out /hive/trunk/ql/src/test/results/clientpositive/join32.q.out /hive/trunk/ql/src/test/results/clientpositive/join33.q.out /hive/trunk/ql/src/test/results/clientpositive/join_star.q.out /hive/trunk/ql/src/test/results/clientpositive/mapjoin_filter_on_outerjoin.q.out /hive/trunk/ql/src/test/results/clientpositive/mapjoin_mapjoin.q.out /hive/trunk/ql/src/test/results/clientpositive/mapjoin_subquery.q.out /hive/trunk/ql/src/test/results/clientpositive/mapjoin_subquery2.q.out /hive/trunk/ql/src/test/results/clientpositive/mapjoin_test_outer.q.out /hive/trunk/ql/src/test/results/clientpositive/multiMapJoin1.q.out /hive/trunk/ql/src/test/results/clientpositive/multiMapJoin2.q.out /hive/trunk/ql/src/test/results/clientpositive/multi_join_union.q.out /hive/trunk/ql/src/test/results/clientpositive/union34.q.out
          Hide
          Ashutosh Chauhan added a comment -

          This issue has been fixed and released as part of 0.12 release. If you find further issues, please create a new jira and link it to this one.

          Show
          Ashutosh Chauhan added a comment - This issue has been fixed and released as part of 0.12 release. If you find further issues, please create a new jira and link it to this one.
          Ashutosh Chauhan made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Yin Huai made changes -
          Link This issue is related to HIVE-5891 [ HIVE-5891 ]

            People

            • Assignee:
              Yin Huai
              Reporter:
              Yin Huai
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development