Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.12.0
    • Component/s: None
    • Labels:
      None

      Description

      HIVE-4675 was merged but there are still a few minor issues we need to cleanup:

      • README is out of date
      • Need to limit the number of failed source directories we copy back from the slaves
      • when looking for TEST-*.xml files we look at both the log directory (good) and the failed source directories (bad) therefore duplicating failures in jenkins report
      • We need to process bad hosts in the finally block of PTest.run (HIVE-4882)
      • Need a mechanism to clean the ivy and maven cache (HIVE-4882)
      • PTest2 fails to publish a comment to a JIRA sometimes (HIVE-4889)
      • Now that PTest2 is committed to the source tree it's copying in our TEST-SomeTest*.xml files

      Test Properties:
      NO PRECOMMIT TESTS

      1. HIVE-4892.patch
        106 kB
        Brock Noland
      2. HIVE-4892.patch
        55 kB
        Brock Noland

        Issue Links

          Activity

          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          ERROR: -1 due to 1 failed/errored test(s), 5 tests executed
          Failed tests:

          failed
          

          Test results: https://builds.apache.org/job/test/123/testReport
          Console output: https://builds.apache.org/job/test/123/console
          Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logstest-123/

          Messages:

          msg1
          msg2
          

          This message is automatically generated.

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed ERROR: -1 due to 1 failed/errored test(s), 5 tests executed Failed tests: failed Test results: https://builds.apache.org/job/test/123/testReport Console output: https://builds.apache.org/job/test/123/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logstest-123/ Messages: msg1 msg2 This message is automatically generated.
          Hide
          Szehon Ho added a comment -

          Overall: -1 at least one tests failed

          ERROR: -1 due to 1 failed/errored test(s), 5 tests executed
          Failed tests:

          failed
          

          Test results: https://builds.apache.org/job/test/123/testReport
          Console output: https://builds.apache.org/job/test/123/console
          Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logstest-123/

          Messages:

          msg1
          msg2
          

          This message is automatically generated.

          Show
          Szehon Ho added a comment - Overall : -1 at least one tests failed ERROR: -1 due to 1 failed/errored test(s), 5 tests executed Failed tests: failed Test results: https://builds.apache.org/job/test/123/testReport Console output: https://builds.apache.org/job/test/123/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logstest-123/ Messages: msg1 msg2 This message is automatically generated.
          Hide
          Ashutosh Chauhan added a comment -

          This issue has been fixed and released as part of 0.12 release. If you find further issues, please create a new jira and link it to this one.

          Show
          Ashutosh Chauhan added a comment - This issue has been fixed and released as part of 0.12 release. If you find further issues, please create a new jira and link it to this one.
          Hide
          Brock Noland added a comment -

          Hey,

          This patch contain some deletes and therefore left some empty files when it was applied. We should execute an addendum commit:

          svn rm ./testutils/ptest2/src/test/resources/test-outputs/TEST-SomeTest-truncated.xml ./testutils/ptest2/src/test/resources/test-outputs/TEST-skewjoin.q-ab8536a7-1b5c-45ed-ba29-14450f27db8b-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml ./testutils/ptest2/src/test/resources/test-outputs/TEST-union_remove_9.q-acb9de8f-1b9c-4874-924c-b2107ca7b07c-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml ./testutils/ptest2/src/test/resources/test-outputs/TEST-skewjoin_union_remove_1.q-6fa31776-d2b0-4e13-9761-11f750627ad1-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml ./testutils/ptest2/src/test/resources/test-outputs/TEST-index_auth.q-bucketcontex-ba31fb54-1d7f-4c70-a89d-477b7d155191-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml ./testutils/ptest2/src/test/resources/TEST-SomeTest-failure.xml
          
          Show
          Brock Noland added a comment - Hey, This patch contain some deletes and therefore left some empty files when it was applied. We should execute an addendum commit: svn rm ./testutils/ptest2/src/test/resources/test-outputs/TEST-SomeTest-truncated.xml ./testutils/ptest2/src/test/resources/test-outputs/TEST-skewjoin.q-ab8536a7-1b5c-45ed-ba29-14450f27db8b-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml ./testutils/ptest2/src/test/resources/test-outputs/TEST-union_remove_9.q-acb9de8f-1b9c-4874-924c-b2107ca7b07c-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml ./testutils/ptest2/src/test/resources/test-outputs/TEST-skewjoin_union_remove_1.q-6fa31776-d2b0-4e13-9761-11f750627ad1-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml ./testutils/ptest2/src/test/resources/test-outputs/TEST-index_auth.q-bucketcontex-ba31fb54-1d7f-4c70-a89d-477b7d155191-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml ./testutils/ptest2/src/test/resources/TEST-SomeTest-failure.xml
          Hide
          Brock Noland added a comment -

          Looks like this has been committed. Thank you!!

          Show
          Brock Noland added a comment - Looks like this has been committed. Thank you!!
          Hide
          Edward Capriolo added a comment -

          committing.

          Show
          Edward Capriolo added a comment - committing.
          Hide
          Brock Noland added a comment -

          Ping

          Show
          Brock Noland added a comment - Ping
          Hide
          Ashutosh Chauhan added a comment -

          +1

          Show
          Ashutosh Chauhan added a comment - +1
          Hide
          Brock Noland added a comment -

          Hi,

          PTest2 parses the TEST-*.xml logs and the patch does include "sample" TEST outputs for uniting testing purposes. Unfortunately in the current version they are named TEST..xml in the source tree which is causing ptest2 issues when it finds these outputs. This patch renames them to remove the TEST prefix. There is also a few very small other ouptuts such as hive.log for the same purpose. These already exist in the source tree but are being renamed.

          When we get this committed I can submit a performance improvement patch that should increase throughput of the pre-commit tests by 2x.

          Show
          Brock Noland added a comment - Hi, PTest2 parses the TEST-*.xml logs and the patch does include "sample" TEST outputs for uniting testing purposes. Unfortunately in the current version they are named TEST..xml in the source tree which is causing ptest2 issues when it finds these outputs. This patch renames them to remove the TEST prefix. There is also a few very small other ouptuts such as hive.log for the same purpose. These already exist in the source tree but are being renamed. When we get this committed I can submit a performance improvement patch that should increase throughput of the pre-commit tests by 2x.
          Hide
          Ashutosh Chauhan added a comment -

          There are few new files in there which looks like test logs. Are those needed ?

          Show
          Ashutosh Chauhan added a comment - There are few new files in there which looks like test logs. Are those needed ?
          Hide
          Brock Noland added a comment -

          Removed and renamed the TEST-*.xml files we use for testing parsing of output files. This was causing issues with the build. These changed make the patch look much larger than it actually is.

          Show
          Brock Noland added a comment - Removed and renamed the TEST-*.xml files we use for testing parsing of output files. This was causing issues with the build. These changed make the patch look much larger than it actually is.
          Hide
          Brock Noland added a comment -

          Friendly ping When we commit this I can move the build infra to the official source tree.

          Show
          Brock Noland added a comment - Friendly ping When we commit this I can move the build infra to the official source tree.
          Show
          Brock Noland added a comment - https://reviews.apache.org/r/12790/
          Hide
          Xuefu Zhang added a comment -

          That will do. Thanks for the effort. I will put that in HIVE-3632 to see how it goes.

          Show
          Xuefu Zhang added a comment - That will do. Thanks for the effort. I will put that in HIVE-3632 to see how it goes.
          Hide
          Brock Noland added a comment -

          Xuefu Zhang

          Here is how I plan to handle clearing the library cache for a build. Basically if you needed the library cache cleaned for a particular build, in the JIRA description (so you can edit it later) you'd say:

          Test Properties:
          CLEAR LIBRARY CACHE

          which would then enable deleting the library cache for that build. Does that sound good?

          Show
          Brock Noland added a comment - Xuefu Zhang Here is how I plan to handle clearing the library cache for a build. Basically if you needed the library cache cleaned for a particular build, in the JIRA description (so you can edit it later) you'd say: Test Properties: CLEAR LIBRARY CACHE which would then enable deleting the library cache for that build. Does that sound good?
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          ERROR: -1 due to 1 failed/errored test(s), 5 tests executed
          Failed tests:

          failed
          

          Test results: https://builds.apache.org/job/test/123/testReport
          Console output: https://builds.apache.org/job/test/123/console

          Messages:

          msg1
          msg2
          

          This message is automatically generated.

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed ERROR: -1 due to 1 failed/errored test(s), 5 tests executed Failed tests: failed Test results: https://builds.apache.org/job/test/123/testReport Console output: https://builds.apache.org/job/test/123/console Messages: msg1 msg2 This message is automatically generated.
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          ERROR: -1 due to 1 failed/errored test(s), 5 tests executed
          Failed tests:

          failed
          

          Test results: https://builds.apache.org/job/test/123/testReport
          Console output: https://builds.apache.org/job/test/123/console

          Messages:

          This message is automatically generated.

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed ERROR: -1 due to 1 failed/errored test(s), 5 tests executed Failed tests: failed Test results: https://builds.apache.org/job/test/123/testReport Console output: https://builds.apache.org/job/test/123/console Messages: This message is automatically generated.
          Hide
          Hive QA added a comment -

          Overall: +1 all checks pass

          SUCCESS: +1 all tests passed

          Test results: https://builds.apache.org/job/test/123/testReport
          Console output: https://builds.apache.org/job/test/123/console

          Messages:
          test"test

          This message is automatically generated.

          Show
          Hive QA added a comment - Overall : +1 all checks pass SUCCESS: +1 all tests passed Test results: https://builds.apache.org/job/test/123/testReport Console output: https://builds.apache.org/job/test/123/console Messages: test"test This message is automatically generated.
          Hide
          Hive QA added a comment -

          \n\nOverall: +1 all checks pass\n\n\nSUCCESS: +1 all tests passed\n\nTest results: https://builds.apache.org/job/test/123/testReport\nConsole output: https://builds.apache.org/job/test/123/console\n\nMessages:\ntest"test\n\nThis message is automatically generated.

          Show
          Hive QA added a comment - \n\n Overall : +1 all checks pass\n\n\n SUCCESS: +1 all tests passed\n\nTest results: https://builds.apache.org/job/test/123/testReport\nConsole output: https://builds.apache.org/job/test/123/console\n\nMessages:\ntest "test\n\nThis message is automatically generated.
          Hide
          Hive QA added a comment -

          Overall: +1 all checks pass

          SUCCESS: +1 all tests passed

          Test results: https://builds.apache.org/job/test/123/testReport
          Console output: https://builds.apache.org/job/test/123/console

          Messages:
          testn'test

          This message is automatically generated.

          Show
          Hive QA added a comment - Overall : +1 all checks pass SUCCESS: +1 all tests passed Test results: https://builds.apache.org/job/test/123/testReport Console output: https://builds.apache.org/job/test/123/console Messages: testn'test This message is automatically generated.
          Hide
          Hive QA added a comment -

          Overall: +1 all checks pass

          SUCCESS: +1 all tests passed

          Test results: https://builds.apache.org/job/test/123/testReport
          Console output: https://builds.apache.org/job/test/123/console

          Messages:
          testn'test'n

          This message is automatically generated.

          Show
          Hive QA added a comment - Overall : +1 all checks pass SUCCESS: +1 all tests passed Test results: https://builds.apache.org/job/test/123/testReport Console output: https://builds.apache.org/job/test/123/console Messages: testn'test'n This message is automatically generated.

            People

            • Assignee:
              Brock Noland
              Reporter:
              Brock Noland
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development