Hive
  1. Hive
  2. HIVE-4877

In ExecReducer, remove tag from the row which will be passed to the first Operator at the Reduce-side

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.12.0
    • Fix Version/s: 0.12.0
    • Component/s: Query Processor
    • Labels:
      None

      Description

      In ExecReducer, we can see ...

      row.clear();
      row.add(keyObject);
      row.add(valueObject[tag.get()]);
      // The tag is not used any more, we should remove it.
      row.add(tag);
      

      I have seen this for a long time. Actually, we are using

      reducer.process(row, tag.get());
      

      to pass the tag.

      If we do not need to add the tag to a row, we should remove it.

      1. HIVE-4877.1.patch
        50 kB
        Yin Huai
      2. HIVE-4877.patch
        50 kB
        Yin Huai

        Issue Links

          Activity

          Hide
          Ashutosh Chauhan added a comment -

          Yeah we should get rid of this tag. This makes code unnecessarily complicated in operators.

          Show
          Ashutosh Chauhan added a comment - Yeah we should get rid of this tag. This makes code unnecessarily complicated in operators.
          Hide
          Yin Huai added a comment -

          Add review board link https://reviews.apache.org/r/12767/.

          My arc diff either failed or pushed to my 2206 page...

          Show
          Yin Huai added a comment - Add review board link https://reviews.apache.org/r/12767/ . My arc diff either failed or pushed to my 2206 page...
          Hide
          Ashutosh Chauhan added a comment -

          Left some comments on RB.

          Show
          Ashutosh Chauhan added a comment - Left some comments on RB.
          Hide
          Yin Huai added a comment -

          Review board has been updated. I am also attaching the patch at here.

          Show
          Yin Huai added a comment - Review board has been updated. I am also attaching the patch at here.
          Hide
          Ashutosh Chauhan added a comment -

          +1

          Show
          Ashutosh Chauhan added a comment - +1
          Hide
          Yin Huai added a comment -

          A minor update. In TestExecDriver, we should use LogFactory.getLog for logging. Seems using LogFactory.getLog is the convention.

          Show
          Yin Huai added a comment - A minor update. In TestExecDriver, we should use LogFactory.getLog for logging. Seems using LogFactory.getLog is the convention.
          Hide
          Yin Huai added a comment -

          HIVE-4877.1 is the latest one

          Show
          Yin Huai added a comment - HIVE-4877 .1 is the latest one
          Hide
          Hive QA added a comment -

          Overall: +1 all checks pass

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12593234/HIVE-4877.patch

          SUCCESS: +1 2646 tests passed

          Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/98/testReport
          Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/98/console

          Messages:

          Executing org.apache.hive.ptest.execution.CleanupPhase
          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          

          This message is automatically generated.

          Show
          Hive QA added a comment - Overall : +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12593234/HIVE-4877.patch SUCCESS: +1 2646 tests passed Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/98/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/98/console Messages: Executing org.apache.hive.ptest.execution.CleanupPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase This message is automatically generated.
          Hide
          Hive QA added a comment -

          Overall: +1 all checks pass

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12593255/HIVE-4877.1.patch

          SUCCESS: +1 2646 tests passed

          Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/100/testReport
          Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/100/console

          Messages:

          Executing org.apache.hive.ptest.execution.CleanupPhase
          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          

          This message is automatically generated.

          Show
          Hive QA added a comment - Overall : +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12593255/HIVE-4877.1.patch SUCCESS: +1 2646 tests passed Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/100/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/100/console Messages: Executing org.apache.hive.ptest.execution.CleanupPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase This message is automatically generated.
          Hide
          Ashutosh Chauhan added a comment -

          Committed to trunk. Thanks, Yin!

          Show
          Ashutosh Chauhan added a comment - Committed to trunk. Thanks, Yin!
          Hide
          Ashutosh Chauhan added a comment -

          This issue has been fixed and released as part of 0.12 release. If you find further issues, please create a new jira and link it to this one.

          Show
          Ashutosh Chauhan added a comment - This issue has been fixed and released as part of 0.12 release. If you find further issues, please create a new jira and link it to this one.

            People

            • Assignee:
              Yin Huai
              Reporter:
              Yin Huai
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development