Details

    • Type: Task Task
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Currently we excluding the following tests:

      unitTests.exclude = TestHiveMetaStore TestSerDe TestBeeLineDriver TestHiveServer2Concurrency TestJdbcDriver2 TestHiveServer2Concurrency TestBeeLineDriver

      some of them we got from the build files but I am not sure about TestJdbcDriver2 for example. We should investigate why these are excluded.

        Issue Links

          Activity

          Hide
          Brock Noland added a comment -

          TestJdbcDriver2 ran just fine, I have enabled it.

          Show
          Brock Noland added a comment - TestJdbcDriver2 ran just fine, I have enabled it.
          Hide
          Ashutosh Chauhan added a comment -

          In recent runs, I noticed Hive QA runs around ~2653 tests while when I run my tests old style (ie via ant test) it reports running ~2692 tests. It will be good to verify the same number of tests are run in either cases.

          Show
          Ashutosh Chauhan added a comment - In recent runs, I noticed Hive QA runs around ~2653 tests while when I run my tests old style (ie via ant test) it reports running ~2692 tests. It will be good to verify the same number of tests are run in either cases.
          Hide
          Brock Noland added a comment -

          Will do

          Show
          Brock Noland added a comment - Will do
          Hide
          Brock Noland added a comment -

          OK, the first item is that PTest2 wasn't reporting on all the tests that it ran. This was fixed in HIVE-4892. Now as it did https://issues.apache.org/jira/browse/HIVE-4299?focusedCommentId=13721929&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13721929 and here PTest2 should be reporting ~2729.

          I counted the number of tests ran by ant test and ptest2 and I got the following This is using grep '<testcase':

          $ wc -l ant.txt ptest.txt 
            2747 ant.txt
            2729 ptest.txt
          

          I found some interesting items.

          1) Ptest2 doesn't exclude skipped tests from reporting
          2) PTest2 tracks "include" properties specified in the build.xml files separately (only impacts minimr and testhbasedriver) and they are slightly out of date. Short term I can fix this and create instructions on how to update PTest2 when adding a minimr test. Medium term we should just parse these properties out of the build.xml.
          3) Some tests don't start with Test and therefore PTest2 misses them
          4) In PTest2 we list out the directories we expect to find unit tests. One directory is missing. Short term fix is to list it and medium term fix is just to look in all directories.

          Show
          Brock Noland added a comment - OK, the first item is that PTest2 wasn't reporting on all the tests that it ran. This was fixed in HIVE-4892 . Now as it did https://issues.apache.org/jira/browse/HIVE-4299?focusedCommentId=13721929&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13721929 and here PTest2 should be reporting ~2729. I counted the number of tests ran by ant test and ptest2 and I got the following This is using grep '<testcase': $ wc -l ant.txt ptest.txt 2747 ant.txt 2729 ptest.txt I found some interesting items. 1) Ptest2 doesn't exclude skipped tests from reporting 2) PTest2 tracks "include" properties specified in the build.xml files separately (only impacts minimr and testhbasedriver) and they are slightly out of date. Short term I can fix this and create instructions on how to update PTest2 when adding a minimr test. Medium term we should just parse these properties out of the build.xml. 3) Some tests don't start with Test and therefore PTest2 misses them 4) In PTest2 we list out the directories we expect to find unit tests. One directory is missing. Short term fix is to list it and medium term fix is just to look in all directories.
          Hide
          Brock Noland added a comment -

          Actually 4. above should be that some tests cannot be executed via ant test -Dtestcase. For example TestHadoop20SAuthBridge.

          Show
          Brock Noland added a comment - Actually 4. above should be that some tests cannot be executed via ant test -Dtestcase. For example TestHadoop20SAuthBridge.
          Hide
          Ashutosh Chauhan added a comment -

          Whats the status of these tasks? Are all tests now running with PTest2 ?

          Show
          Ashutosh Chauhan added a comment - Whats the status of these tasks? Are all tests now running with PTest2 ?
          Hide
          Brock Noland added a comment -

          I believe so but i'd be good for me to do one more confirmation.

          Show
          Brock Noland added a comment - I believe so but i'd be good for me to do one more confirmation.
          Hide
          Brock Noland added a comment -

          FYI I am running a local ant test and will compare the results tomorrow.

          Show
          Brock Noland added a comment - FYI I am running a local ant test and will compare the results tomorrow.
          Hide
          Brock Noland added a comment -

          And my local run failed.

          Show
          Brock Noland added a comment - And my local run failed.
          Hide
          Anthony Hsu added a comment -

          I'm trying to run TestHiveMetaStore but I'm having trouble getting it to run. I made the following changes to the root pom.xml file

          diff --git a/pom.xml b/pom.xml
          index 5d4f13c..a8098ce 100644
          --- a/pom.xml
          +++ b/pom.xml
          @@ -712,12 +712,12 @@
                   <configuration>
                     <excludes>
                       <exclude>**/TestSerDe.java</exclude>
          -            <exclude>**/TestHiveMetaStore.java</exclude>
          +            <!--exclude>**/TestHiveMetaStore.java</exclude-->
                       <exclude>**/ql/exec/vector/util/*.java</exclude>
                       <exclude>**/ql/exec/vector/udf/legacy/*.java</exclude>
                       <exclude>**/ql/exec/vector/udf/generic/*.java</exclude>
                       <exclude>**/TestHiveServer2Concurrency.java</exclude>
          -            <exclude>**/TestHiveMetaStore.java</exclude>
          +            <!--exclude>**/TestHiveMetaStore.java</exclude-->
                     </excludes>
                     <redirectTestOutputToFile>true</redirectTestOutputToFile>
                     <reuseForks>false</reuseForks>
          

          and then ran

          mvn clean install -DskipTests -Phadoop-2
          cd itests
          mvn clean install -DskipTests -Phadoop-2
          mvn test -Phadoop-2 -Dtest=TestHiveMetaStore
          

          However, in the output, I see no tests were run:

          -------------------------------------------------------
           T E S T S
          -------------------------------------------------------
          
          Results :
          
          Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
          

          A couple questions:

          1. Why is the TestHiveMetaStore test excluded?
          2. How do I run the TestHiveMetaStore test?
          Show
          Anthony Hsu added a comment - I'm trying to run TestHiveMetaStore but I'm having trouble getting it to run. I made the following changes to the root pom.xml file diff --git a/pom.xml b/pom.xml index 5d4f13c..a8098ce 100644 --- a/pom.xml +++ b/pom.xml @@ -712,12 +712,12 @@ <configuration> <excludes> <exclude>**/TestSerDe.java</exclude> - <exclude>**/TestHiveMetaStore.java</exclude> + <!--exclude>**/TestHiveMetaStore.java</exclude--> <exclude>**/ql/exec/vector/util/*.java</exclude> <exclude>**/ql/exec/vector/udf/legacy/*.java</exclude> <exclude>**/ql/exec/vector/udf/ generic /*.java</exclude> <exclude>**/TestHiveServer2Concurrency.java</exclude> - <exclude>**/TestHiveMetaStore.java</exclude> + <!--exclude>**/TestHiveMetaStore.java</exclude--> </excludes> <redirectTestOutputToFile> true </redirectTestOutputToFile> <reuseForks> false </reuseForks> and then ran mvn clean install -DskipTests -Phadoop-2 cd itests mvn clean install -DskipTests -Phadoop-2 mvn test -Phadoop-2 -Dtest=TestHiveMetaStore However, in the output, I see no tests were run: ------------------------------------------------------- T E S T S ------------------------------------------------------- Results : Tests run: 0, Failures: 0, Errors: 0, Skipped: 0 A couple questions: Why is the TestHiveMetaStore test excluded? How do I run the TestHiveMetaStore test?
          Hide
          Anthony Hsu added a comment -

          I figured it out. I'm not supposed to run TestHiveMetaStore directly. I'm supposed to run the subclasses TestEmbeddedHiveMetaStore and TestRemoteHiveMetaStore instead. Running the subclasses works.

          Show
          Anthony Hsu added a comment - I figured it out. I'm not supposed to run TestHiveMetaStore directly. I'm supposed to run the subclasses TestEmbeddedHiveMetaStore and TestRemoteHiveMetaStore instead. Running the subclasses works.

            People

            • Assignee:
              Brock Noland
              Reporter:
              Brock Noland
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development