Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.3, 2.0.2-alpha
    • Fix Version/s: 3.0.0, 2.0.3-alpha, 0.23.5
    • Component/s: test
    • Labels:
      None

      Description

      When analyzing code coverage in Hadoop Core, we noticed that some coverage gaps are caused by the way the coverage calculation is done currently. More specifically, right now coverage can not be calculated for the whole Core at once, but can only be calculated separately for top level modules like common-project, hadoop-hdfs-project etc.

      At the same time, some code in particular modules is tested by tests in other modules of Core. For example, "org.apache.hadoop.fs" from hadoop-common-project/hadoop-common is not covered there but it's covered by tests under hadoop-hdfs-project.

      To enable calculation of "cumulative" code coverage it's needed to move Clover profile definition up one level, from hadoop-project/pom.xml to the top level pom.xml (hadoop-main).

      Patch both for 0.23 and 2.x will be attached shortly.

      1. HADOOP-8930-branch-0.23.patch
        5 kB
        Andrey Klochkov
      2. HADOOP-8930.patch
        6 kB
        Andrey Klochkov

        Activity

        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1229 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1229/)
        HADOOP-8930. Cumulative code coverage calculation (Andrey Klochkov via bobby) (Revision 1399426)

        Result = FAILURE
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399426
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-project/pom.xml
        • /hadoop/common/trunk/pom.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1229 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1229/ ) HADOOP-8930 . Cumulative code coverage calculation (Andrey Klochkov via bobby) (Revision 1399426) Result = FAILURE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399426 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-project/pom.xml /hadoop/common/trunk/pom.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1199 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1199/)
        HADOOP-8930. Cumulative code coverage calculation (Andrey Klochkov via bobby) (Revision 1399426)

        Result = SUCCESS
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399426
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-project/pom.xml
        • /hadoop/common/trunk/pom.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1199 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1199/ ) HADOOP-8930 . Cumulative code coverage calculation (Andrey Klochkov via bobby) (Revision 1399426) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399426 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-project/pom.xml /hadoop/common/trunk/pom.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #408 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/408/)
        HADOOP-8930. Cumulative code coverage calculation (Andrey Klochkov via bobby) (Revision 1399428)

        Result = FAILURE
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399428
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml
        • /hadoop/common/branches/branch-0.23/pom.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #408 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/408/ ) HADOOP-8930 . Cumulative code coverage calculation (Andrey Klochkov via bobby) (Revision 1399428) Result = FAILURE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399428 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml /hadoop/common/branches/branch-0.23/pom.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Yarn-trunk #7 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/7/)
        HADOOP-8930. Cumulative code coverage calculation (Andrey Klochkov via bobby) (Revision 1399426)

        Result = FAILURE
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399426
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-project/pom.xml
        • /hadoop/common/trunk/pom.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Yarn-trunk #7 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/7/ ) HADOOP-8930 . Cumulative code coverage calculation (Andrey Klochkov via bobby) (Revision 1399426) Result = FAILURE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399426 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-project/pom.xml /hadoop/common/trunk/pom.xml
        Hide
        Robert Joseph Evans added a comment -

        Thanks Andrey,

        I pulled this into trunk, branch-2, and branch-0.23.

        Show
        Robert Joseph Evans added a comment - Thanks Andrey, I pulled this into trunk, branch-2, and branch-0.23.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-trunk-Commit #2883 (See https://builds.apache.org/job/Hadoop-trunk-Commit/2883/)
        HADOOP-8930. Cumulative code coverage calculation (Andrey Klochkov via bobby) (Revision 1399426)

        Result = SUCCESS
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399426
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-project/pom.xml
        • /hadoop/common/trunk/pom.xml
        Show
        Hudson added a comment - Integrated in Hadoop-trunk-Commit #2883 (See https://builds.apache.org/job/Hadoop-trunk-Commit/2883/ ) HADOOP-8930 . Cumulative code coverage calculation (Andrey Klochkov via bobby) (Revision 1399426) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399426 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-project/pom.xml /hadoop/common/trunk/pom.xml
        Hide
        Robert Joseph Evans added a comment -

        The change looks fine to me, Clover seems to build and run at least as well as it did before. I am +1 for the change.

        Show
        Robert Joseph Evans added a comment - The change looks fine to me, Clover seems to build and run at least as well as it did before. I am +1 for the change.
        Hide
        Andrey Klochkov added a comment -

        Now new tests are needed as the patch updates build configuration only.
        Also seems the robot doesn't apply non-trunk patches to a proper branch.

        Show
        Andrey Klochkov added a comment - Now new tests are needed as the patch updates build configuration only. Also seems the robot doesn't apply non-trunk patches to a proper branch.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12549394/HADOOP-8930-branch-0.23.patch
        against trunk revision .

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1638//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12549394/HADOOP-8930-branch-0.23.patch against trunk revision . -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1638//console This message is automatically generated.
        Hide
        Andrey Klochkov added a comment -

        Attaching a patch for 0.23

        Show
        Andrey Klochkov added a comment - Attaching a patch for 0.23
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12549388/HADOOP-8930.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1637//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1637//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12549388/HADOOP-8930.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in . +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1637//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1637//console This message is automatically generated.
        Hide
        Andrey Klochkov added a comment -

        Attaching a patch for 2.x/trunk

        Show
        Andrey Klochkov added a comment - Attaching a patch for 2.x/trunk

          People

          • Assignee:
            Andrey Klochkov
            Reporter:
            Andrey Klochkov
          • Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development