Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-783

libhdfs tests brakes code coverage runs with Clover

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.21.0
    • Component/s: test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      libhdfs test is executed by a script which sets a certain environment for every run. While standalone execution works well, code coverage runs are broken by this test because it tries to executed an instrumented Hadoop code without having clover.jar in its classpath.

      1. HDFS-783.patch
        1 kB
        Konstantin Boudnik
      2. HDFS-783.patch
        1 kB
        Konstantin Boudnik
      3. HDFS-783-2.patch
        0.8 kB
        Konstantin Boudnik
      4. HDFS-783-2.patch
        0.5 kB
        Konstantin Boudnik

        Activity

        Hide
        Konstantin Boudnik added a comment -

        The patch adds clover.jar file to the classpath before cluster components needed for libhdfs test are started.

        Show
        Konstantin Boudnik added a comment - The patch adds clover.jar file to the classpath before cluster components needed for libhdfs test are started.
        Hide
        Eli Collins added a comment -

        Patch looks good. Btw the libhdfs test isn't new, same one that's been checked in for a while.

        Show
        Eli Collins added a comment - Patch looks good. Btw the libhdfs test isn't new, same one that's been checked in for a while.
        Hide
        Konstantin Boudnik added a comment -

        Turns out that I've done the patch for the outdated version of the shell script. This new version is the same as before but put one of the lines to a different spot.

        Also, I've ran test-patch locally and all seems to be ok:

        There appear to be 125 release audit warnings before the patch and 125 release audit warnings after applying the patch.
        
        +1 overall.
            +1 @author.  The patch does not contain any @author tags.
            +1 tests included.  The patch appears to include 4 new or modified tests.
            +1 javadoc.  The javadoc tool did not generate any warning messages.
            +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
            +1 findbugs.  The patch does not introduce any new Findbugs warnings.
            +1 release audit.  The applied patch does not increase the total number of release audit warnings.
        

        I've started ant clover test run locally and now it passes without a glitch, i.e. c++ compilation is went through and libhdfs test has passed.

        Show
        Konstantin Boudnik added a comment - Turns out that I've done the patch for the outdated version of the shell script. This new version is the same as before but put one of the lines to a different spot. Also, I've ran test-patch locally and all seems to be ok: There appear to be 125 release audit warnings before the patch and 125 release audit warnings after applying the patch. +1 overall. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. I've started ant clover test run locally and now it passes without a glitch, i.e. c++ compilation is went through and libhdfs test has passed.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #121 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/121/)
        . libhdfs tests brakes code coverage runs with Clover. Contributed by Konstantin Boudnik

        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #121 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/121/ ) . libhdfs tests brakes code coverage runs with Clover. Contributed by Konstantin Boudnik
        Hide
        Konstantin Boudnik added a comment -

        I've just committed this.

        Show
        Konstantin Boudnik added a comment - I've just committed this.
        Hide
        Hudson added a comment -

        Integrated in Hdfs-Patch-h5.grid.sp2.yahoo.net #123 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/123/)
        . libhdfs tests brakes code coverage runs with Clover. Contributed by Konstantin Boudnik

        Show
        Hudson added a comment - Integrated in Hdfs-Patch-h5.grid.sp2.yahoo.net #123 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/123/ ) . libhdfs tests brakes code coverage runs with Clover. Contributed by Konstantin Boudnik
        Hide
        Konstantin Boudnik added a comment -

        Still doesn't work in Hudson. Apparently, just adding clover.jar into the classpath doesn't solve the issue with script.

        Show
        Konstantin Boudnik added a comment - Still doesn't work in Hudson. Apparently, just adding clover.jar into the classpath doesn't solve the issue with script.
        Hide
        Konstantin Boudnik added a comment -

        Further investigation shows that test-libhdfs indirectly calls hadoop-config.sh which resets $CLASSPATH settings. Thus, it seems to be quite impossible or fix this issue with the current approach to the testing of libhdfs

        The test has to be disabled until it is done without direct calls to Hadoop's startup scripts.

        Show
        Konstantin Boudnik added a comment - Further investigation shows that test-libhdfs indirectly calls hadoop-config.sh which resets $CLASSPATH settings. Thus, it seems to be quite impossible or fix this issue with the current approach to the testing of libhdfs The test has to be disabled until it is done without direct calls to Hadoop's startup scripts.
        Hide
        Konstantin Boudnik added a comment -

        This patch will completely disable the execution of test-c++-libhdfs if Clover is enabled, e.g. in code coverage runs. On the other hand, it will still run this test on non-instrumented code. It is a dirty hack but otherwise there's no way to run full code coverage build.

        Show
        Konstantin Boudnik added a comment - This patch will completely disable the execution of test-c++-libhdfs if Clover is enabled, e.g. in code coverage runs. On the other hand, it will still run this test on non-instrumented code. It is a dirty hack but otherwise there's no way to run full code coverage build.
        Hide
        Konstantin Boudnik added a comment -

        Adding an info message to the clover.setup target.

        Show
        Konstantin Boudnik added a comment - Adding an info message to the clover.setup target.
        Hide
        Nigel Daley added a comment -

        +1 code review

        Show
        Nigel Daley added a comment - +1 code review
        Hide
        Konstantin Boudnik added a comment -

        Have committed the second take of the fix.

        Show
        Konstantin Boudnik added a comment - Have committed the second take of the fix.
        Hide
        Hudson added a comment -

        Integrated in Hdfs-Patch-h5.grid.sp2.yahoo.net #127 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/127/)

        Show
        Hudson added a comment - Integrated in Hdfs-Patch-h5.grid.sp2.yahoo.net #127 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/127/ )
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #128 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/128/)

        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #128 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/128/ )
        Hide
        Hudson added a comment -

        Integrated in Hdfs-Patch-h5.grid.sp2.yahoo.net #129 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/129/)

        Show
        Hudson added a comment - Integrated in Hdfs-Patch-h5.grid.sp2.yahoo.net #129 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/129/ )
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #159 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk/159/)

        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #159 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk/159/ )
        Hide
        Hudson added a comment -

        Integrated in Hdfs-Patch-h2.grid.sp2.yahoo.net #81 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/81/)

        Show
        Hudson added a comment - Integrated in Hdfs-Patch-h2.grid.sp2.yahoo.net #81 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/81/ )

          People

          • Assignee:
            Konstantin Boudnik
            Reporter:
            Konstantin Boudnik
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development