Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-4968

When deduplicating multiple SelectOperators, we should update RowResolver accordinly

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.12.0
    • Component/s: None
    • Labels:
      None

      Description

      SELECT tmp3.key, tmp3.value, tmp3.count
      FROM (SELECT tmp1.key as key, tmp1.value as value, tmp2.count as count
            FROM (SELECT key, value
                  FROM src) tmp1
            JOIN (SELECT count(*) as count
                  FROM src) tmp2
            ) tmp3;
      

      The plan is executable.

      SELECT tmp3.key, tmp3.value, tmp3.count
      FROM (SELECT tmp1.key as key, tmp1.value as value, tmp2.count as count
            FROM (SELECT *
                  FROM src) tmp1
            JOIN (SELECT count(*) as count
                  FROM src) tmp2
            ) tmp3;
      

      The plan is executable.

      SELECT tmp4.key, tmp4.value, tmp4.count
      FROM (SELECT tmp2.key as key, tmp2.value as value, tmp3.count as count
            FROM (SELECT *
                  FROM (SELECT key, value
                        FROM src) tmp1 ) tmp2
            JOIN (SELECT count(*) as count
                  FROM src) tmp3
            ) tmp4;
      

      The plan is not executable.

      The plan related to the MapJoin is

       Stage: Stage-5
          Map Reduce Local Work
            Alias -> Map Local Tables:
              tmp4:tmp2:tmp1:src 
                Fetch Operator
                  limit: -1
            Alias -> Map Local Operator Tree:
              tmp4:tmp2:tmp1:src 
                TableScan
                  alias: src
                  Select Operator
                    expressions:
                          expr: key
                          type: string
                          expr: value
                          type: string
                    outputColumnNames: _col0, _col1
                    HashTable Sink Operator
                      condition expressions:
                        0 
                        1 {_col0}
                      handleSkewJoin: false
                      keys:
                        0 []
                        1 []
                      Position of Big Table: 1
      
        Stage: Stage-4
          Map Reduce
            Alias -> Map Operator Tree:
              $INTNAME 
                  Map Join Operator
                    condition map:
                         Inner Join 0 to 1
                    condition expressions:
                      0 
                      1 {_col0}
                    handleSkewJoin: false
                    keys:
                      0 []
                      1 []
                    outputColumnNames: _col2
                    Position of Big Table: 1
                    Select Operator
                      expressions:
                            expr: _col0
                            type: string
                            expr: _col1
                            type: string
                            expr: _col2
                            type: bigint
                      outputColumnNames: _col0, _col1, _col2
                      File Output Operator
                        compressed: false
                        GlobalTableId: 0
                        table:
                            input format: org.apache.hadoop.mapred.TextInputFormat
                            output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
            Local Work:
              Map Reduce Local Work
      

      The outputColumnNames of MapJoin is '_col2'. But it should be '_col0, _col1, _col2'

        Attachments

        1. HIVE-4968.D11901.1.patch
          18 kB
          Phabricator
        2. HIVE-4968.D11901.2.patch
          20 kB
          Phabricator

          Activity

            People

            • Assignee:
              yhuai Yin Huai
              Reporter:
              yhuai Yin Huai
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: