Hadoop Common
  1. Hadoop Common
  2. HADOOP-2813

Unit test fails on Linux: org.apache.hadoop.fs.TestDU.testDU

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.16.1
    • Fix Version/s: 0.16.1
    • Component/s: fs
    • Labels:
      None

      Description

      Unit test fails on Linux: org.apache.hadoop.fs.TestDU.testDU. This is a regression

      Changes that went in yesterday were:
      HADOOP-2725. Modify distcp to avoid leaving partially copied files at the destination after encountering an error.
      HADOOP-2193. 'fs -rm' and 'fs -rmr' show error message when the target file does not exist.

      Here is the error from the test run:
      org.apache.hadoop.util.Shell$ExitCodeException: /usr/bin/du: cannot access `/home/xxxxxx/workspace/Hadoop-LinuxTest/trunk/build/test/mapred/system/distcp_i0ebq9/.nfs00000000004aa47f00001ba0': No such file or directory
      /usr/bin/du: cannot access `/home/xxxxxx/workspace/Hadoop-LinuxTest/trunk/build/test/mapred/system/distcp_i0ebq9/.nfs00000000004aa48000001b9f': No such file or directory
      /usr/bin/du: cannot access `/home/xxxxxx/workspace/Hadoop-LinuxTest/trunk/build/test/mapred/system/distcp_r3r1fo/.nfs00000000004aa48200001ba2': No such file or directory
      /usr/bin/du: cannot access `/home/xxxxxx/workspace/Hadoop-LinuxTest/trunk/build/test/mapred/system/distcp_r3r1fo/.nfs00000000004aa48300001ba1': No such file or directory

      at org.apache.hadoop.util.Shell.runCommand(Shell.java:161)
      at org.apache.hadoop.util.Shell.run(Shell.java:100)
      at org.apache.hadoop.fs.DU.getUsed(DU.java:53)
      at org.apache.hadoop.fs.TestDU.testDU(TestDU.java:62)
      at org.apache.hadoop.fs.TestDU.testDU(TestDU.java:71)

      1. hadoop-2813_1.patch
        0.9 kB
        Mahadev konar
      2. hadoop-2813_2.patch
        1 kB
        Mahadev konar

        Issue Links

          Activity

          Hide
          Hudson added a comment -
          Show
          Hudson added a comment - Integrated in Hadoop-trunk #424 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/424/ )
          Hide
          Nigel Daley added a comment -

          This problem existed in the 0.16 branch as well so I merged it to that branch.

          Show
          Nigel Daley added a comment - This problem existed in the 0.16 branch as well so I merged it to that branch.
          Hide
          Hudson added a comment -
          Show
          Hudson added a comment - Integrated in Hadoop-trunk #405 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/405/ )
          Hide
          dhruba borthakur added a comment -

          I just resolved this. Thanks Mahadev!

          Show
          dhruba borthakur added a comment - I just resolved this. Thanks Mahadev!
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12375705/hadoop-2813_2.patch
          against trunk revision 619744.

          @author +1. The patch does not contain any @author tags.

          tests included +1. The patch appears to include 3 new or modified tests.

          javadoc +1. The javadoc tool did not generate any warning messages.

          javac +1. The applied patch does not generate any new javac compiler warnings.

          release audit +1. The applied patch does not generate any new release audit warnings.

          findbugs +1. The patch does not introduce any new Findbugs warnings.

          core tests +1. The patch passed core unit tests.

          contrib tests +1. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1805/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1805/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1805/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1805/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12375705/hadoop-2813_2.patch against trunk revision 619744. @author +1. The patch does not contain any @author tags. tests included +1. The patch appears to include 3 new or modified tests. javadoc +1. The javadoc tool did not generate any warning messages. javac +1. The applied patch does not generate any new javac compiler warnings. release audit +1. The applied patch does not generate any new release audit warnings. findbugs +1. The patch does not introduce any new Findbugs warnings. core tests +1. The patch passed core unit tests. contrib tests +1. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1805/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1805/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1805/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1805/console This message is automatically generated.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          +1

          Show
          Tsz Wo Nicholas Sze added a comment - +1
          Hide
          Mahadev konar added a comment -

          this patch deletes the directory after the test is complete.

          Show
          Mahadev konar added a comment - this patch deletes the directory after the test is complete.
          Hide
          Mukund Madhugiri added a comment -

          Test passes with the patch. Thanks Mahadev

          Show
          Mukund Madhugiri added a comment - Test passes with the patch. Thanks Mahadev
          Hide
          Tsz Wo Nicholas Sze added a comment -

          The problem is probably caused by TestCopyFiles. It might be the case that TestCopyFiles deletes some files which are not closed properly. So that the OS creates some temporary .nfs files. TestDU sees these files in the beginning of the test (since it is reading the test root directory). These files are cleaned up by the OS, therefore, TestDU cannot see them again and says "No such file or directory".

          A more correct way is to fix TestCopyFiles so that all files are closed probably. However, since the codes in TestCopyFiles are complicated, it is much easier to fix it by changing the test directory in TestDU, as done by Mahadev. We should fix TestCopyFiles later.

          Comment on hadoop-2813_1.patch:
          Codes are good. Could you clean up DU_DIR at the end of the test?

          Show
          Tsz Wo Nicholas Sze added a comment - The problem is probably caused by TestCopyFiles. It might be the case that TestCopyFiles deletes some files which are not closed properly. So that the OS creates some temporary .nfs files. TestDU sees these files in the beginning of the test (since it is reading the test root directory). These files are cleaned up by the OS, therefore, TestDU cannot see them again and says "No such file or directory". A more correct way is to fix TestCopyFiles so that all files are closed probably. However, since the codes in TestCopyFiles are complicated, it is much easier to fix it by changing the test directory in TestDU, as done by Mahadev. We should fix TestCopyFiles later. Comment on hadoop-2813_1.patch: Codes are good. Could you clean up DU_DIR at the end of the test?
          Hide
          Mahadev konar added a comment -

          mukund can you try with this patch. DU uses current working directory as the test directory. I fixed it to use its own directory under build. This does not fix the creation of tmp .nfs files but should pass the tests.

          I am unable to recreate the problem on my machine.

          Show
          Mahadev konar added a comment - mukund can you try with this patch. DU uses current working directory as the test directory. I fixed it to use its own directory under build. This does not fix the creation of tmp .nfs files but should pass the tests. I am unable to recreate the problem on my machine.
          Hide
          Mukund Madhugiri added a comment -

          This is definitely repeatable. The test fails consistently

          Show
          Mukund Madhugiri added a comment - This is definitely repeatable. The test fails consistently
          Hide
          Mukund Madhugiri added a comment -

          This was on trunk, not 0.16.1

          Show
          Mukund Madhugiri added a comment - This was on trunk, not 0.16.1

            People

            • Assignee:
              Mahadev konar
              Reporter:
              Mukund Madhugiri
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development