Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.13.0
    • Component/s: None
    • Labels:
      None

      Description

      In order to facilitate merge back I've separated out all the changes that don't require Tez. These changes introduce new interfaces, move code etc. In preparation of the Tez specific classes. This should help show what changes have been made that affect the MR codepath as well.

      1. HIVE-6125.4.patch
        3.10 MB
        Gunther Hagleitner
      2. HIVE-6125.3.patch
        3.10 MB
        Gunther Hagleitner
      3. HIVE-6125.2.patch
        3.10 MB
        Gunther Hagleitner
      4. HIVE-6125.1.patch
        3.10 MB
        Gunther Hagleitner

        Issue Links

          Activity

          Hide
          Gunther Hagleitner added a comment -

          Still large. Mostly due to the fact that the TestParse xml files have changed...

          Show
          Gunther Hagleitner added a comment - Still large. Mostly due to the fact that the TestParse xml files have changed...
          Show
          Gunther Hagleitner added a comment - RB: https://reviews.apache.org/r/16557/
          Hide
          Lefty Leverenz added a comment -

          Patch 1 adds three config params to HiveConf.java:

          • hive.orc.splits.include.file.footer
          • hive.orc.cache.stripe.details.size
          • hive.orc.compute.splits.num.threads

          But it doesn't define them in hive-default.xml.template. Just sayin'.

          Show
          Lefty Leverenz added a comment - Patch 1 adds three config params to HiveConf.java: hive.orc.splits.include.file.footer hive.orc.cache.stripe.details.size hive.orc.compute.splits.num.threads But it doesn't define them in hive-default.xml.template. Just sayin'.
          Hide
          Gunther Hagleitner added a comment -

          Thanks for the feedback Lefty Leverenz. I'll create new jira for that. Definitely useful to have them in hive-default.xml as another means of documentation.

          Show
          Gunther Hagleitner added a comment - Thanks for the feedback Lefty Leverenz . I'll create new jira for that. Definitely useful to have them in hive-default.xml as another means of documentation.
          Hide
          Gunther Hagleitner added a comment -

          Captured in HIVE-6128

          Show
          Gunther Hagleitner added a comment - Captured in HIVE-6128
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12621026/HIVE-6125.1.patch

          ERROR: -1 due to 1 failed/errored test(s), 4873 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_mapreduce_stack_trace_hadoop20
          

          Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/787/testReport
          Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/787/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 1 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12621026

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12621026/HIVE-6125.1.patch ERROR: -1 due to 1 failed/errored test(s), 4873 tests executed Failed tests: org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_mapreduce_stack_trace_hadoop20 Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/787/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/787/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed This message is automatically generated. ATTACHMENT ID: 12621026
          Hide
          Gunther Hagleitner added a comment -

          failure seems unrelated. passes locally.

          Show
          Gunther Hagleitner added a comment - failure seems unrelated. passes locally.
          Hide
          Gunther Hagleitner added a comment -

          This one is without golden files (easier to navigate): https://reviews.apache.org/r/16611/

          Show
          Gunther Hagleitner added a comment - This one is without golden files (easier to navigate): https://reviews.apache.org/r/16611/
          Hide
          Gunther Hagleitner added a comment -

          No change in .2 other than rebasing it.

          Show
          Gunther Hagleitner added a comment - No change in .2 other than rebasing it.
          Hide
          Gunther Hagleitner added a comment -

          .3 is another rebase

          Show
          Gunther Hagleitner added a comment - .3 is another rebase
          Hide
          Thejas M Nair added a comment -

          LGTM. Just some minor comments in review board, otherwise +1.

          Show
          Thejas M Nair added a comment - LGTM. Just some minor comments in review board, otherwise +1.
          Hide
          Gunther Hagleitner added a comment -

          .4 addresses the review comments.

          Show
          Gunther Hagleitner added a comment - .4 addresses the review comments.
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12621386/HIVE-6125.3.patch

          ERROR: -1 due to 1 failed/errored test(s), 4874 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket_num_reducers
          

          Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/802/testReport
          Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/802/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 1 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12621386

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12621386/HIVE-6125.3.patch ERROR: -1 due to 1 failed/errored test(s), 4874 tests executed Failed tests: org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket_num_reducers Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/802/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/802/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed This message is automatically generated. ATTACHMENT ID: 12621386
          Hide
          Hive QA added a comment -

          Overall: +1 all checks pass

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12621408/HIVE-6125.4.patch

          SUCCESS: +1 4875 tests passed

          Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/804/testReport
          Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/804/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          

          This message is automatically generated.

          ATTACHMENT ID: 12621408

          Show
          Hive QA added a comment - Overall : +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12621408/HIVE-6125.4.patch SUCCESS: +1 4875 tests passed Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/804/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/804/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase This message is automatically generated. ATTACHMENT ID: 12621408
          Hide
          Gunther Hagleitner added a comment -

          Committed to trunk.

          Show
          Gunther Hagleitner added a comment - Committed to trunk.
          Hide
          Lefty Leverenz added a comment -

          This adds three hive.orc.* configuration parameters to HiveConf.java: hive.orc.splits.include.file.footer, hive.orc.cache.stripe.details.size, and hive.orc.compute.splits.num.threads.

          Their descriptions are added in HIVE-6128 and they are documented in the wiki here:

          Show
          Lefty Leverenz added a comment - This adds three hive.orc.* configuration parameters to HiveConf.java: hive.orc.splits.include.file.footer , hive.orc.cache.stripe.details.size , and hive.orc.compute.splits.num.threads . Their descriptions are added in HIVE-6128 and they are documented in the wiki here: Configuration Properties – hive.orc.splits.include.file.footer Configuration Properties – hive.orc.cache.stripe.details.size Configuration Properties – hive.orc.compute.splits.num.threads

            People

            • Assignee:
              Gunther Hagleitner
              Reporter:
              Gunther Hagleitner
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development