Hive
  1. Hive
  2. HIVE-4927

When we merge two MapJoin MapRedTasks, the TableScanOperator of the second one should be removed

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.12.0
    • Fix Version/s: 0.12.0
    • Component/s: Query Processor
    • Labels:
      None

      Description

      set hive.auto.convert.join=true;
      set hive.auto.convert.join.noconditionaltask=true;
      EXPLAIN
      SELECT x1.key AS key FROM src x1 JOIN src1 y1 ON (x1.key = y1.key) JOIN src1 y2 ON (x1.value = y2.value) GROUP BY x1.key;
      

      We will get a NPE from MetadataOnlyOptimizer. The reason is that the operator tree of the MapRedTask evaluating two MapJoins is

      TS1->MapJoin1->TS2->MapJoin2->...
      

      We should remove the TS2...

      1. HIVE-4927.D11811.1.patch
        17 kB
        Phabricator
      2. HIVE-4927.D11811.2.patch
        186 kB
        Phabricator
      3. HIVE-4927.D11811.3.patch
        74 kB
        Yin Huai
      4. HIVE-4927.D11811.3.patch
        74 kB
        Phabricator

        Issue Links

          Activity

          Ashutosh Chauhan made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Ashutosh Chauhan made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s 0.12.0 [ 12324312 ]
          Resolution Fixed [ 1 ]
          Yin Huai made changes -
          Attachment HIVE-4927.D11811.3.patch [ 12594308 ]
          Yin Huai made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Yin Huai made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Yin Huai made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Phabricator made changes -
          Attachment HIVE-4927.D11811.3.patch [ 12594200 ]
          Phabricator made changes -
          Attachment HIVE-4927.D11811.2.patch [ 12594199 ]
          Yin Huai made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Phabricator made changes -
          Attachment HIVE-4927.D11811.1.patch [ 12594067 ]
          Yin Huai made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Yin Huai made changes -
          Remote Link This issue links to "Phabricator (Web Link)" [ 12439 ]
          Yin Huai made changes -
          Link This issue relates to HIVE-4827 [ HIVE-4827 ]
          Yin Huai made changes -
          Link This issue is related to HIVE-4872 [ HIVE-4872 ]
          Yin Huai made changes -
          Link This issue is related to HIVE-4872 [ HIVE-4872 ]
          Yin Huai made changes -
          Attachment HIVE-4927.1.patch [ 12594036 ]
          Yin Huai made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Yin Huai made changes -
          Affects Version/s 0.12.0 [ 12324312 ]
          Yin Huai made changes -
          Component/s Query Processor [ 12312586 ]
          Yin Huai made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Yin Huai made changes -
          Field Original Value New Value
          Attachment HIVE-4927.1.patch [ 12594036 ]
          Yin Huai created issue -

            People

            • Assignee:
              Yin Huai
              Reporter:
              Yin Huai
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development