Details

    • Type: Test Test
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.9.0, 0.10.0, 0.11.0
    • Fix Version/s: None
    • Component/s: Testing Infrastructure
    • Labels:
      None

      Description

      Atlassian has donated license of their code coverage tool Clover to ASF. Lets make use of it to generate code coverage report to figure out which areas of Hive are well tested and which ones are not. More information about license can be found in Hadoop jira HADOOP-1718

      1. ASF.LICENSE.NOT.GRANTED--HIVE-2991.D2985.1.patch
        5 kB
        Phabricator
      2. hive.2991.1.branch-0.10.patch
        7 kB
        Ilya Katsov
      3. hive.2991.1.branch-0.9.patch
        7 kB
        Ilya Katsov
      4. hive.2991.1.trunk.patch
        7 kB
        Ilya Katsov
      5. hive.2991.2.branch-0.10.patch
        9 kB
        Ilya Katsov
      6. hive.2991.2.branch-0.9.patch
        10 kB
        Ilya Katsov
      7. hive.2991.2.trunk.patch
        9 kB
        Ilya Katsov
      8. hive.2991.3.branch-0.10.patch
        11 kB
        Ilya Katsov
      9. hive.2991.3.branch-0.9.patch
        11 kB
        Ilya Katsov
      10. hive.2991.3.trunk.patch
        12 kB
        Ilya Katsov
      11. hive.2991.4.branch-0.10.patch
        11 kB
        Ilya Katsov
      12. hive.2991.4.branch-0.9.patch
        12 kB
        Ilya Katsov
      13. hive.2991.4.trunk.patch
        13 kB
        Ilya Katsov
      14. HIVE-clover-branch-0.10--N1.patch
        61 kB
        Ivan A. Veselovsky
      15. HIVE-clover-branch-0.11--N1.patch
        70 kB
        Ivan A. Veselovsky
      16. HIVE-clover-trunk--N1.patch
        71 kB
        Ivan A. Veselovsky
      17. hive-trunk-clover-html-report.zip
        342 kB
        Ilya Katsov

        Activity

        Ashutosh Chauhan created issue -
        Hide
        Phabricator added a comment -

        ashutoshc requested code review of "HIVE-2991 [jira] Integrate Clover with Hive".
        Reviewers: JIRA

        https://issues.apache.org/jira/browse/HIVE-2991

        This patch integrates clover with Hive. To generate code coverage follow these steps:

        Download the clover jar and license from https://svn.apache.org/repos/private/committers/donated-licenses/clover/2.6.x/README.txt and put them in same dir
        (lets say to /home/me/clover/). Then to generate the clover reports run ant clean package test clover-reports -Dclover.jar=/home/me/clover/clover.jar

        TEST PLAN: Run ant clean package test clover-reports -Dclover.jar=/home/me/clover/clover.jar and reports should be in build/$

        {ant.project.name}

        /clover/reports

        Atlassian has donated license of their code coverage tool Clover to ASF. Lets make use of it to generate code coverage report to figure out which areas of Hive are well tested and which ones are not. More information about license can be found in Hadoop jira HADOOP-1718

        TEST PLAN
        EMPTY

        REVISION DETAIL
        https://reviews.facebook.net/D2985

        AFFECTED FILES
        build.xml
        build-common.xml

        MANAGE HERALD DIFFERENTIAL RULES
        https://reviews.facebook.net/herald/view/differential/

        WHY DID I GET THIS EMAIL?
        https://reviews.facebook.net/herald/transcript/6795/

        Tip: use the X-Herald-Rules header to filter Herald messages in your client.

        Show
        Phabricator added a comment - ashutoshc requested code review of " HIVE-2991 [jira] Integrate Clover with Hive". Reviewers: JIRA https://issues.apache.org/jira/browse/HIVE-2991 This patch integrates clover with Hive. To generate code coverage follow these steps: Download the clover jar and license from https://svn.apache.org/repos/private/committers/donated-licenses/clover/2.6.x/README.txt and put them in same dir (lets say to /home/me/clover/). Then to generate the clover reports run ant clean package test clover-reports -Dclover.jar=/home/me/clover/clover.jar TEST PLAN: Run ant clean package test clover-reports -Dclover.jar=/home/me/clover/clover.jar and reports should be in build/$ {ant.project.name} /clover/reports Atlassian has donated license of their code coverage tool Clover to ASF. Lets make use of it to generate code coverage report to figure out which areas of Hive are well tested and which ones are not. More information about license can be found in Hadoop jira HADOOP-1718 TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D2985 AFFECTED FILES build.xml build-common.xml MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/6795/ Tip: use the X-Herald-Rules header to filter Herald messages in your client.
        Phabricator made changes -
        Field Original Value New Value
        Attachment HIVE-2991.D2985.1.patch [ 12525141 ]
        Hide
        Edward Capriolo added a comment -

        I would be interested to see this. The crazy way we use velocity templates to generate out unit tests had me thinking that clover would not work well. I would like to be pleasantly surprised.

        Show
        Edward Capriolo added a comment - I would be interested to see this. The crazy way we use velocity templates to generate out unit tests had me thinking that clover would not work well. I would like to be pleasantly surprised.
        Hide
        Ashutosh Chauhan added a comment -

        Here is the report I got by running it on hive trunk of last week: http://people.apache.org/~hashutosh/hive-clover/

        Show
        Ashutosh Chauhan added a comment - Here is the report I got by running it on hive trunk of last week: http://people.apache.org/~hashutosh/hive-clover/
        Hide
        Edward Capriolo added a comment -

        It is a nice report, as I expected I am not sure the coverage accurately depicts reality http://people.apache.org/~hashutosh/hive-clover/ql/. You would think that all the clientpositive directory would give better coverage. Still a good thing to have and improve on.

        Show
        Edward Capriolo added a comment - It is a nice report, as I expected I am not sure the coverage accurately depicts reality http://people.apache.org/~hashutosh/hive-clover/ql/ . You would think that all the clientpositive directory would give better coverage. Still a good thing to have and improve on.
        Ilya Katsov made changes -
        Attachment hive.2991.1.branch-0.9.patch [ 12556278 ]
        Attachment hive.2991.1.branch-0.10.patch [ 12556279 ]
        Ilya Katsov made changes -
        Attachment hive.2991.1.trunk.patch [ 12556280 ]
        Ilya Katsov made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Affects Version/s 0.9.0 [ 12317742 ]
        Hide
        Ilya Katsov added a comment -

        Apparently it makes sense to exclude auto generated and sample code from the coverage reports to obtain realistic total coverage percentage. Updated patches has been attached.

        Show
        Ilya Katsov added a comment - Apparently it makes sense to exclude auto generated and sample code from the coverage reports to obtain realistic total coverage percentage. Updated patches has been attached.
        Hide
        Ashutosh Chauhan added a comment -

        Ilya Katsov Can you link the report that you got by running clover on recent hive trunk?

        Show
        Ashutosh Chauhan added a comment - Ilya Katsov Can you link the report that you got by running clover on recent hive trunk?
        Hide
        Ilya Katsov added a comment -

        Latest report for trunk is attached (class-level info was removed to reduce the archive size).

        Show
        Ilya Katsov added a comment - Latest report for trunk is attached (class-level info was removed to reduce the archive size).
        Ilya Katsov made changes -
        Attachment hive-trunk-clover-html-report.zip [ 12560730 ]
        Ilya Katsov made changes -
        Attachment hive.2991.2.branch-0.9.patch [ 12560763 ]
        Attachment hive.2991.2.branch-0.10.patch [ 12560764 ]
        Attachment hive.2991.2.trunk.patch [ 12560765 ]
        Hide
        Daisy.Yuan added a comment -

        when run “ ant clean package test clover-reports -Dclover.jar=/opt/yoo/clover-2.6.2.jar”, it meets the following exception:
        /opt/trunk/build.xml:264: The following error occurred while executing this line:
        /opt/trunk/build-common.xml:126: java.lang.RuntimeException: Invalid or missing License.. Please visit http://my.atlassian.com to obtain a valid license.
        at com.cenqua.clover.CloverStartup.loadLicense(CloverStartup.java:58)
        at com.cenqua.clover.CloverStartup.loadLicense(CloverStartup.java:25)
        at com.cenqua.clover.tasks.AbstractCloverTask.execute(AbstractCloverTask.java:52)
        at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
        at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
        at org.apache.tools.ant.Task.perform(Task.java:348)
        at org.apache.tools.ant.Target.execute(Target.java:392)
        at org.apache.tools.ant.Target.performTasks(Target.java:413)
        at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
        at org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38)
        at org.apache.tools.ant.Project.executeTargets(Project.java:1251)
        at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:442)
        at org.apache.tools.ant.taskdefs.SubAnt.execute(SubAnt.java:303)
        at org.apache.tools.ant.taskdefs.SubAnt.execute(SubAnt.java:221)
        at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
        at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
        at org.apache.tools.ant.Task.perform(Task.java:348)
        at org.apache.tools.ant.Target.execute(Target.java:392)
        at org.apache.tools.ant.Target.performTasks(Target.java:413)
        at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
        at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
        at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
        at org.apache.tools.ant.Project.executeTargets(Project.java:1251)
        at org.apache.tools.ant.Main.runBuild(Main.java:811)
        at org.apache.tools.ant.Main.startAnt(Main.java:217)
        at org.apache.tools.ant.launch.Launcher.run(Launcher.java:280)
        at org.apache.tools.ant.launch.Launcher.main(Launcher.java:109)

        Total time: 2 seconds

        The clover-2.6.x.jar needs license, and I get a try-it-free license, and add it to the Dclover.jar path. try it again, it still failed because of the above reason.

        Does it have any free jar to replace the clover.jar

        Show
        Daisy.Yuan added a comment - when run “ ant clean package test clover-reports -Dclover.jar=/opt/yoo/clover-2.6.2.jar”, it meets the following exception: /opt/trunk/build.xml:264: The following error occurred while executing this line: /opt/trunk/build-common.xml:126: java.lang.RuntimeException: Invalid or missing License.. Please visit http://my.atlassian.com to obtain a valid license. at com.cenqua.clover.CloverStartup.loadLicense(CloverStartup.java:58) at com.cenqua.clover.CloverStartup.loadLicense(CloverStartup.java:25) at com.cenqua.clover.tasks.AbstractCloverTask.execute(AbstractCloverTask.java:52) at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291) at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106) at org.apache.tools.ant.Task.perform(Task.java:348) at org.apache.tools.ant.Target.execute(Target.java:392) at org.apache.tools.ant.Target.performTasks(Target.java:413) at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399) at org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38) at org.apache.tools.ant.Project.executeTargets(Project.java:1251) at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:442) at org.apache.tools.ant.taskdefs.SubAnt.execute(SubAnt.java:303) at org.apache.tools.ant.taskdefs.SubAnt.execute(SubAnt.java:221) at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291) at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106) at org.apache.tools.ant.Task.perform(Task.java:348) at org.apache.tools.ant.Target.execute(Target.java:392) at org.apache.tools.ant.Target.performTasks(Target.java:413) at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399) at org.apache.tools.ant.Project.executeTarget(Project.java:1368) at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41) at org.apache.tools.ant.Project.executeTargets(Project.java:1251) at org.apache.tools.ant.Main.runBuild(Main.java:811) at org.apache.tools.ant.Main.startAnt(Main.java:217) at org.apache.tools.ant.launch.Launcher.run(Launcher.java:280) at org.apache.tools.ant.launch.Launcher.main(Launcher.java:109) Total time: 2 seconds The clover-2.6.x.jar needs license, and I get a try-it-free license, and add it to the Dclover.jar path. try it again, it still failed because of the above reason. Does it have any free jar to replace the clover.jar
        Hide
        Andrey Klochkov added a comment -

        Daisy, you need to use "clover.license.path" JVM parameter. This is how we build:

        ant clean package test clover-reports -Dtest.silent=false -Dtest.ignore.failures=true -Dtest.junit.output.format=xml -Dclover.jar=$CLOVER_HOME/lib/clover.jar -Dclover.license.path=$HOME/.clover.license
        
        Show
        Andrey Klochkov added a comment - Daisy, you need to use "clover.license.path" JVM parameter. This is how we build: ant clean package test clover-reports -Dtest.silent= false -Dtest.ignore.failures= true -Dtest.junit.output.format=xml -Dclover.jar=$CLOVER_HOME/lib/clover.jar -Dclover.license.path=$HOME/.clover.license
        Hide
        Carl Steinbach added a comment -

        @Ashutosh: I left some comments on phabricator. Thanks.

        Show
        Carl Steinbach added a comment - @Ashutosh: I left some comments on phabricator. Thanks.
        Carl Steinbach made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Hide
        Phabricator added a comment -

        cwsteinbach has requested changes to the revision "HIVE-2991 [jira] Integrate Clover with Hive".

        INLINE COMMENTS
        build-common.xml:112 indentation

        build-common.xml:125 indentation
        build-common.xml:118 We should exclude generated code (e.g. metastore/src/gen, metastore/src/model) from this list.
        build-common.xml:115 This should check to see if clover.license.path and clover.jar are set, and print an informative error message if they're missing.

        REVISION DETAIL
        https://reviews.facebook.net/D2985

        BRANCH
        svn

        To: JIRA, cwsteinbach, ashutoshc

        Show
        Phabricator added a comment - cwsteinbach has requested changes to the revision " HIVE-2991 [jira] Integrate Clover with Hive". INLINE COMMENTS build-common.xml:112 indentation build-common.xml:125 indentation build-common.xml:118 We should exclude generated code (e.g. metastore/src/gen, metastore/src/model) from this list. build-common.xml:115 This should check to see if clover.license.path and clover.jar are set, and print an informative error message if they're missing. REVISION DETAIL https://reviews.facebook.net/D2985 BRANCH svn To: JIRA, cwsteinbach, ashutoshc
        Hide
        Ashish Vaidya added a comment -

        I'm new to Ant/Ivy based builds. Please pardon my ignorance. Could someone please explain what's the rationale behind excluding com.cenqua.clover in ivy.xml and then using -Dclover.jar to pass in the Clover JAR location? I understand -Dclover.license.path to be configurable as each user is supposed to have his/her own Clover license.

        What could go wrong if we have <dependency org="com.cenqua.clover" name="clover" rev="3.1.8"/> in the ivy.xml instead of that 'exclude?' Is this JAR not compliant/compatible with Apache License? AFAIK, maven-clover-plugin uses this same JAR and is used in other Apache Hadoop projects.

        Show
        Ashish Vaidya added a comment - I'm new to Ant/Ivy based builds. Please pardon my ignorance. Could someone please explain what's the rationale behind excluding com.cenqua.clover in ivy.xml and then using -Dclover.jar to pass in the Clover JAR location? I understand -Dclover.license.path to be configurable as each user is supposed to have his/her own Clover license. What could go wrong if we have <dependency org="com.cenqua.clover" name="clover" rev="3.1.8"/> in the ivy.xml instead of that 'exclude?' Is this JAR not compliant/compatible with Apache License? AFAIK, maven-clover-plugin uses this same JAR and is used in other Apache Hadoop projects.
        Hide
        Ashutosh Chauhan added a comment -

        I am not actively working on this one. Ashish,Ilya or any one else whoever is interested, feel free to pick it up, if interested.

        Show
        Ashutosh Chauhan added a comment - I am not actively working on this one. Ashish,Ilya or any one else whoever is interested, feel free to pick it up, if interested.
        Ashutosh Chauhan made changes -
        Assignee Ashutosh Chauhan [ ashutoshc ]
        Hide
        Ilya Katsov added a comment -

        Indentations are fixed in hive.2991.2.* patches.

        Generated code is excluded in build.xml as a part of report generation (see clover.report.filest fileset).

        clover.jar property is used to enable clover, clover will be inactive if this property missed. clover.license.path is a clover's internal property and it is not mandatory if the license is placed in the clover's lib directory. If the license is missed, clover prints the error message itself.

        Show
        Ilya Katsov added a comment - Indentations are fixed in hive.2991.2.* patches. Generated code is excluded in build.xml as a part of report generation (see clover.report.filest fileset). clover.jar property is used to enable clover, clover will be inactive if this property missed. clover.license.path is a clover's internal property and it is not mandatory if the license is placed in the clover's lib directory. If the license is missed, clover prints the error message itself.
        Ilya Katsov made changes -
        Attachment hive.2991.2.trunk.patch [ 12560765 ]
        Ilya Katsov made changes -
        Attachment hive.2991.2.trunk.patch [ 12567971 ]
        Ilya Katsov made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Ilya Katsov made changes -
        Attachment hive.2991.3.branch-0.10.patch [ 12569326 ]
        Attachment hive.2991.3.branch-0.9.patch [ 12569327 ]
        Ilya Katsov made changes -
        Attachment hive.2991.3.trunk.patch [ 12569328 ]
        Ilya Katsov made changes -
        Attachment hive.2991.4.branch-0.9.patch [ 12576728 ]
        Attachment hive.2991.4.branch-0.10.patch [ 12576729 ]
        Ilya Katsov made changes -
        Attachment hive.2991.4.trunk.patch [ 12576730 ]
        Hide
        Ivan A. Veselovsky added a comment -

        The attached patches "HIVE-clover-xxx.patch" are somewhat updated versions of the clovering. We used them in parallel builds.

        Except clovering itself the patches introduce the following changes:
        1) .q files test generator changed to split test classes by groups of tests (10 test cases) per class is the default. This is needed to avoid huge test classes – needed for parralalized and distributed builds.
        2) added "test-lightweight" target that allows to run a batch of tests without re-generation/re-compilation. This is badly needed in parallelized and distributed builds.
        3) we introduce "testcase-list" parameter that allows to pass several test class names to execute. The names are to be passed in form of comma-separated list with each name in the list being in form "*/a/b/c/TestFoo.". Last asterisk is needed because main project accepts .class names, while HCatalog accepts .java names.
        4) + several more improvements related to clover instrumentation, reporting, etc.

        Show
        Ivan A. Veselovsky added a comment - The attached patches "HIVE-clover-xxx.patch" are somewhat updated versions of the clovering. We used them in parallel builds. Except clovering itself the patches introduce the following changes: 1) .q files test generator changed to split test classes by groups of tests (10 test cases) per class is the default. This is needed to avoid huge test classes – needed for parralalized and distributed builds. 2) added "test-lightweight" target that allows to run a batch of tests without re-generation/re-compilation. This is badly needed in parallelized and distributed builds. 3) we introduce "testcase-list" parameter that allows to pass several test class names to execute. The names are to be passed in form of comma-separated list with each name in the list being in form "* /a/b/c/TestFoo. ". Last asterisk is needed because main project accepts .class names, while HCatalog accepts .java names. 4) + several more improvements related to clover instrumentation, reporting, etc.
        Ivan A. Veselovsky made changes -
        Attachment HIVE-clover-branch-0.10--N1.patch [ 12591861 ]
        Attachment HIVE-clover-branch-0.11--N1.patch [ 12591862 ]
        Attachment HIVE-clover-trunk--N1.patch [ 12591863 ]
        Ivan A. Veselovsky made changes -
        Assignee Ivan A. Veselovsky [ iveselovsky ]
        Hide
        Ashutosh Chauhan added a comment -

        Ivan A. Veselovsky Seems like you have expanded the scope of this jira quite a bit. Your other changes (introducing targets in build system) are quite useful, but they are orthogonal to clover integration (as far as i understand). I would suggest to split the patch in three parts: one for clover integration, second for improvement in test infrastructure and third for improvements in build infra.

        Show
        Ashutosh Chauhan added a comment - Ivan A. Veselovsky Seems like you have expanded the scope of this jira quite a bit. Your other changes (introducing targets in build system) are quite useful, but they are orthogonal to clover integration (as far as i understand). I would suggest to split the patch in three parts: one for clover integration, second for improvement in test infrastructure and third for improvements in build infra.
        Hide
        Ashutosh Chauhan added a comment -

        Canceling patch for now. Breaking it down in series of patches (instead of clubbing everything together) for different issues is a good idea.

        Show
        Ashutosh Chauhan added a comment - Canceling patch for now. Breaking it down in series of patches (instead of clubbing everything together) for different issues is a good idea.
        Ashutosh Chauhan made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Affects Version/s 0.11.0 [ 12323587 ]
        Affects Version/s 0.10.0 [ 12320745 ]

          People

          • Assignee:
            Ivan A. Veselovsky
            Reporter:
            Ashutosh Chauhan
          • Votes:
            0 Vote for this issue
            Watchers:
            11 Start watching this issue

            Dates

            • Created:
              Updated:

              Development