Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-1548

Fault-injection tests are executed multiple times if invoked with run-test-hdfs-fault-inject target

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.21.1
    • Fix Version/s: 0.21.1
    • Component/s: build, test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      When invoked with run-test-hdfs-fault-inject target fault injection tests are getting executed 4 times.

      1. HDFS-1548.patch
        0.9 kB
        Konstantin Boudnik
      2. HDFS-1548.patch
        1 kB
        Konstantin Boudnik
      3. HDFS-1548.patch
        1 kB
        Konstantin Boudnik

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-22-branch #35 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-22-branch/35/)

          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-22-branch #35 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-22-branch/35/ )
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #643 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/643/)

          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #643 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/643/ )
          Hide
          Konstantin Boudnik added a comment -

          I have committed this to 0.21 and above.

          Show
          Konstantin Boudnik added a comment - I have committed this to 0.21 and above.
          Hide
          Konstantin Boudnik added a comment -

          Oops, have uploaded a patch different that I had tested and reported here. This version fixes a typo in a target name.

          Show
          Konstantin Boudnik added a comment - Oops, have uploaded a patch different that I had tested and reported here. This version fixes a typo in a target name.
          Hide
          Jakob Homan added a comment -

          +1

          Show
          Jakob Homan added a comment - +1
          Hide
          Konstantin Boudnik added a comment -

          Turns out to be needing a slight more work because -Dtestcase wasn't serving properly. This patch addresses this case as well.

          Also, as a reference point: I have ran ant run-test-hdfs-fault-inject with this result:

          run-test-hdfs-excluding-commit-and-smoke:
             [delete] Deleting directory /home/cos/work/H0.23/git/hdfs/build-fi/test/data
              [mkdir] Created dir: /home/cos/work/H0.23/git/hdfs/build-fi/test/data
             [delete] Deleting directory /home/cos/work/H0.23/git/hdfs/build-fi/test/logs
              [mkdir] Created dir: /home/cos/work/H0.23/git/hdfs/build-fi/test/logs
               [copy] Copying 1 file to /home/cos/work/H0.23/git/hdfs/build-fi/test/extraconf
               [copy] Copying 1 file to /home/cos/work/H0.23/git/hdfs/build-fi/test/extraconf
              [junit] WARNING: multiple versions of ant detected in path for junit
              [junit]          jar:file:/usr/share/ant/lib/ant.jar!/org/apache/tools/ant/Project.class
              [junit]      and jar:file:/home/cos/.ivy2/cache/ant/ant/jars/ant-1.6.5.jar!/org/apache/tools/ant/Project.class
              [junit] Running org.apache.hadoop.fs.TestFiListPath
              [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 4.103 sec
              [junit] Running org.apache.hadoop.fs.TestFiRename
              [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 20.911 sec
              [junit] Running org.apache.hadoop.hdfs.TestFiHFlush
              [junit] Tests run: 9, Failures: 0, Errors: 0, Time elapsed: 35 sec
              [junit] Running org.apache.hadoop.hdfs.TestFiHftp
              [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 38.938 sec
              [junit] Running org.apache.hadoop.hdfs.TestFiPipelines
              [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 9.108 sec
              [junit] Running org.apache.hadoop.hdfs.server.datanode.TestFiDataTransferProtocol
              [junit] Tests run: 29, Failures: 0, Errors: 0, Time elapsed: 282.474 sec
              [junit] Running org.apache.hadoop.hdfs.server.datanode.TestFiDataTransferProtocol2
              [junit] Tests run: 10, Failures: 0, Errors: 0, Time elapsed: 453.364 sec
              [junit] Running org.apache.hadoop.hdfs.server.datanode.TestFiPipelineClose
              [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 44.702 sec
          
          checkfailure:
          
          BUILD SUCCESSFUL
          Total time: 16 minutes 14 seconds
          

          Total build time is about 16 minutes as it should be.

          and the same with -Dtestcase=TestFiHFlush has produced an expected single test execution:

          run-test-hdfs-all-withtestcaseonly:
             [delete] Deleting directory /home/cos/work/H0.23/git/hdfs/build-fi/test/data
              [mkdir] Created dir: /home/cos/work/H0.23/git/hdfs/build-fi/test/data
             [delete] Deleting directory /home/cos/work/H0.23/git/hdfs/build-fi/test/logs
              [mkdir] Created dir: /home/cos/work/H0.23/git/hdfs/build-fi/test/logs
              [junit] WARNING: multiple versions of ant detected in path for junit 
              [junit]          jar:file:/usr/share/ant/lib/ant.jar!/org/apache/tools/ant/Project.class
              [junit]      and jar:file:/home/cos/.ivy2/cache/ant/ant/jars/ant-1.6.5.jar!/org/apache/tools/ant/Project.class
              [junit] Running org.apache.hadoop.hdfs.TestFiHFlush
              [junit] Tests run: 9, Failures: 0, Errors: 0, Time elapsed: 38.937 sec
          
          checkfailure:
          
          run-test-hdfs-fault-inject:
          
          BUILD SUCCESSFUL
          Total time: 1 minute 11 seconds
          
          Show
          Konstantin Boudnik added a comment - Turns out to be needing a slight more work because -Dtestcase wasn't serving properly. This patch addresses this case as well. Also, as a reference point: I have ran ant run-test-hdfs-fault-inject with this result: run-test-hdfs-excluding-commit-and-smoke: [delete] Deleting directory /home/cos/work/H0.23/git/hdfs/build-fi/test/data [mkdir] Created dir: /home/cos/work/H0.23/git/hdfs/build-fi/test/data [delete] Deleting directory /home/cos/work/H0.23/git/hdfs/build-fi/test/logs [mkdir] Created dir: /home/cos/work/H0.23/git/hdfs/build-fi/test/logs [copy] Copying 1 file to /home/cos/work/H0.23/git/hdfs/build-fi/test/extraconf [copy] Copying 1 file to /home/cos/work/H0.23/git/hdfs/build-fi/test/extraconf [junit] WARNING: multiple versions of ant detected in path for junit [junit] jar:file:/usr/share/ant/lib/ant.jar!/org/apache/tools/ant/Project.class [junit] and jar:file:/home/cos/.ivy2/cache/ant/ant/jars/ant-1.6.5.jar!/org/apache/tools/ant/Project.class [junit] Running org.apache.hadoop.fs.TestFiListPath [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 4.103 sec [junit] Running org.apache.hadoop.fs.TestFiRename [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 20.911 sec [junit] Running org.apache.hadoop.hdfs.TestFiHFlush [junit] Tests run: 9, Failures: 0, Errors: 0, Time elapsed: 35 sec [junit] Running org.apache.hadoop.hdfs.TestFiHftp [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 38.938 sec [junit] Running org.apache.hadoop.hdfs.TestFiPipelines [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 9.108 sec [junit] Running org.apache.hadoop.hdfs.server.datanode.TestFiDataTransferProtocol [junit] Tests run: 29, Failures: 0, Errors: 0, Time elapsed: 282.474 sec [junit] Running org.apache.hadoop.hdfs.server.datanode.TestFiDataTransferProtocol2 [junit] Tests run: 10, Failures: 0, Errors: 0, Time elapsed: 453.364 sec [junit] Running org.apache.hadoop.hdfs.server.datanode.TestFiPipelineClose [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 44.702 sec checkfailure: BUILD SUCCESSFUL Total time: 16 minutes 14 seconds Total build time is about 16 minutes as it should be. and the same with -Dtestcase=TestFiHFlush has produced an expected single test execution: run-test-hdfs-all-withtestcaseonly: [delete] Deleting directory /home/cos/work/H0.23/git/hdfs/build-fi/test/data [mkdir] Created dir: /home/cos/work/H0.23/git/hdfs/build-fi/test/data [delete] Deleting directory /home/cos/work/H0.23/git/hdfs/build-fi/test/logs [mkdir] Created dir: /home/cos/work/H0.23/git/hdfs/build-fi/test/logs [junit] WARNING: multiple versions of ant detected in path for junit [junit] jar:file:/usr/share/ant/lib/ant.jar!/org/apache/tools/ant/Project.class [junit] and jar:file:/home/cos/.ivy2/cache/ant/ant/jars/ant-1.6.5.jar!/org/apache/tools/ant/Project.class [junit] Running org.apache.hadoop.hdfs.TestFiHFlush [junit] Tests run: 9, Failures: 0, Errors: 0, Time elapsed: 38.937 sec checkfailure: run-test-hdfs-fault-inject: BUILD SUCCESSFUL Total time: 1 minute 11 seconds
          Hide
          Konstantin Boudnik added a comment -

          This is because that target uses special exclude list
          exclude.list.id="commit.smoke.exclude.list.id"

          Show
          Konstantin Boudnik added a comment - This is because that target uses special exclude list exclude.list.id="commit.smoke.exclude.list.id"
          Hide
          Todd Lipcon added a comment -

          hmm, can you explain this patch? I'm confused by the first hunk - why doesn't that cause the smoke and commit tests to not get run with FI?

          Show
          Todd Lipcon added a comment - hmm, can you explain this patch? I'm confused by the first hunk - why doesn't that cause the smoke and commit tests to not get run with FI?
          Hide
          Konstantin Boudnik added a comment -

          Fix for the issue

          Show
          Konstantin Boudnik added a comment - Fix for the issue

            People

            • Assignee:
              Konstantin Boudnik
              Reporter:
              Konstantin Boudnik
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development