Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-4615

Fix TestDFSShell failures on Windows

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0
    • Fix Version/s: 2.1.0-beta
    • Component/s: test
    • Labels:
    • Environment:

      Windows

    • Hadoop Flags:
      Reviewed

      Description

      List of failed tests with exceptions attached.

      1. HADOOP-9398.patch
        7 kB
        Arpit Agarwal
      2. HDFS-4615.patch
        7 kB
        Arpit Agarwal
      3. TestDFSShell-Failed-testcases-Windows.txt
        4 kB
        Arpit Agarwal

        Issue Links

          Activity

          Hide
          Suresh Srinivas added a comment -

          I merged the patch to branch-2.

          Show
          Suresh Srinivas added a comment - I merged the patch to branch-2.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Sid and Suresh, thanks for fixing it.

          Show
          Tsz Wo Nicholas Sze added a comment - Sid and Suresh, thanks for fixing it.
          Hide
          Suresh Srinivas added a comment -

          Updated the fix version.

          Show
          Suresh Srinivas added a comment - Updated the fix version.
          Hide
          Siddharth Seth added a comment -

          Reverted. Can someone please update the Fix version. I don't seem to have access to do this.

          Show
          Siddharth Seth added a comment - Reverted. Can someone please update the Fix version. I don't seem to have access to do this.
          Hide
          Siddharth Seth added a comment -

          Reverting the branch-2 merge to fix the branch build.

          Show
          Siddharth Seth added a comment - Reverting the branch-2 merge to fix the branch build.
          Hide
          Suresh Srinivas added a comment -

          Tsz Wo Nicholas Sze none of the windows related changes are merged into branch-2 yet. Please revert the merge to branch-2.

          Show
          Suresh Srinivas added a comment - Tsz Wo Nicholas Sze none of the windows related changes are merged into branch-2 yet. Please revert the merge to branch-2.
          Hide
          Dennis Y added a comment -

          this patch applied to branch-2 causes compilation failure:
          Compilation failure: Compilation failure:
          2013-03-25 13:04:41.459 [ERROR] /home/jenkins/jenkins/workspace/Hadoop-common-Integration-branch-2-gd/yhadoop-common/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java:[895,15] WINDOWS is not public in org.apache.hadoop.fs.Path; cannot be accessed from outside package
          2013-03-25 13:04:41.459 [ERROR] /home/jenkins/jenkins/workspace/Hadoop-common-Integration-branch-2-gd/yhadoop-common/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/qjournal/TestNNWithQJM.java:[102,20] WINDOWS is not public in org.apache.hadoop.fs.Path; cannot be accessed from outside package
          2013-03-25 13:04:41.460 [ERROR] -> [Help 1]

          =====================
          class hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Path.java
          contains attribute 'WINDOWS' which is not public in branch-2:
          static final boolean WINDOWS

          Show
          Dennis Y added a comment - this patch applied to branch-2 causes compilation failure: Compilation failure: Compilation failure: 2013-03-25 13:04:41.459 [ERROR] /home/jenkins/jenkins/workspace/Hadoop-common-Integration-branch-2-gd/yhadoop-common/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java: [895,15] WINDOWS is not public in org.apache.hadoop.fs.Path; cannot be accessed from outside package 2013-03-25 13:04:41.459 [ERROR] /home/jenkins/jenkins/workspace/Hadoop-common-Integration-branch-2-gd/yhadoop-common/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/qjournal/TestNNWithQJM.java: [102,20] WINDOWS is not public in org.apache.hadoop.fs.Path; cannot be accessed from outside package 2013-03-25 13:04:41.460 [ERROR] -> [Help 1] ===================== class hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Path.java contains attribute 'WINDOWS' which is not public in branch-2: static final boolean WINDOWS
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1380 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1380/)
          HDFS-4615. Fix TestDFSShell failures on Windows. Contributed by Arpit Agarwal (Revision 1459586)

          Result = SUCCESS
          szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1459586
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1380 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1380/ ) HDFS-4615 . Fix TestDFSShell failures on Windows. Contributed by Arpit Agarwal (Revision 1459586) Result = SUCCESS szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1459586 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1352 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1352/)
          HDFS-4615. Fix TestDFSShell failures on Windows. Contributed by Arpit Agarwal (Revision 1459586)

          Result = FAILURE
          szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1459586
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1352 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1352/ ) HDFS-4615 . Fix TestDFSShell failures on Windows. Contributed by Arpit Agarwal (Revision 1459586) Result = FAILURE szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1459586 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Yarn-trunk #163 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/163/)
          HDFS-4615. Fix TestDFSShell failures on Windows. Contributed by Arpit Agarwal (Revision 1459586)

          Result = FAILURE
          szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1459586
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
          Show
          Hudson added a comment - Integrated in Hadoop-Yarn-trunk #163 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/163/ ) HDFS-4615 . Fix TestDFSShell failures on Windows. Contributed by Arpit Agarwal (Revision 1459586) Result = FAILURE szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1459586 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-trunk-Commit #3508 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3508/)
          HDFS-4615. Fix TestDFSShell failures on Windows. Contributed by Arpit Agarwal (Revision 1459586)

          Result = SUCCESS
          szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1459586
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
          Show
          Hudson added a comment - Integrated in Hadoop-trunk-Commit #3508 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3508/ ) HDFS-4615 . Fix TestDFSShell failures on Windows. Contributed by Arpit Agarwal (Revision 1459586) Result = SUCCESS szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1459586 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
          Hide
          Arpit Agarwal added a comment -

          Thanks Nicholas!

          Show
          Arpit Agarwal added a comment - Thanks Nicholas!
          Hide
          Tsz Wo Nicholas Sze added a comment -

          I have committed this. Thanks, Aprit!

          Also, thanks Chris for reviewing this.

          Show
          Tsz Wo Nicholas Sze added a comment - I have committed this. Thanks, Aprit! Also, thanks Chris for reviewing this.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12574574/HDFS-4615.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 tests included appear to have a timeout.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/4122//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4122//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12574574/HDFS-4615.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 tests included appear to have a timeout. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/4122//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4122//console This message is automatically generated.
          Hide
          Chris Nauroth added a comment -

          +1 for the patch. Thanks for addressing the feedback.

          Show
          Chris Nauroth added a comment - +1 for the patch. Thanks for addressing the feedback.
          Hide
          Arpit Agarwal added a comment -

          Thanks Chris! Updated patch attached.

          Show
          Arpit Agarwal added a comment - Thanks Chris! Updated patch attached.
          Hide
          Chris Nauroth added a comment -

          Nice finds, Arpit! I verified that this works on Mac and Windows. Here are just a couple of minor clean-up comments:

          +      // Skip sticky but related tests on Windows.
          +      //
          

          Minor typo: change to "sticky bit". Maybe delete the second blank comment line.

          +      System.out.println("trashFile is " + trashFile);
          

          Can we remove this now?

          Show
          Chris Nauroth added a comment - Nice finds, Arpit! I verified that this works on Mac and Windows. Here are just a couple of minor clean-up comments: + // Skip sticky but related tests on Windows. + // Minor typo: change to "sticky bit". Maybe delete the second blank comment line. + System .out.println( "trashFile is " + trashFile); Can we remove this now?
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12574472/HADOOP-9398.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 tests included appear to have a timeout.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/4119//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4119//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12574472/HADOOP-9398.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 tests included appear to have a timeout. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/4119//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4119//console This message is automatically generated.
          Hide
          Arpit Agarwal added a comment -

          (All fixes in the test cases)

          FS shell commands expect valid URIs as inputs and hence fails when given Windows absolute paths. While it is easy enough to fix it to take paths I do not want to change that behavior. Hence the path related fixes are all in the test cases.

          Show
          Arpit Agarwal added a comment - (All fixes in the test cases) FS shell commands expect valid URIs as inputs and hence fails when given Windows absolute paths. While it is easy enough to fix it to take paths I do not want to change that behavior. Hence the path related fixes are all in the test cases.
          Hide
          Arpit Agarwal added a comment -

          Attaching list of failed test cases.

          Show
          Arpit Agarwal added a comment - Attaching list of failed test cases.

            People

            • Assignee:
              Arpit Agarwal
              Reporter:
              Arpit Agarwal
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 8h
                8h
                Remaining:
                Remaining Estimate - 8h
                8h
                Logged:
                Time Spent - Not Specified
                Not Specified

                  Development