Hadoop Common
  1. Hadoop Common
  2. HADOOP-9105

FsShell -moveFromLocal erroneously fails

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0, 2.0.0-alpha, 3.0.0
    • Fix Version/s: 3.0.0, 2.0.3-alpha, 0.23.6
    • Component/s: fs
    • Labels:
      None

      Description

      The move successfully completes, but then reports error upon trying to delete the local source directory even though it succeeded.

      1. HADOOP-9105.branch-0.23.patch
        5 kB
        Daryn Sharp
      2. HADOOP-9105.patch
        5 kB
        Daryn Sharp

        Activity

        Hide
        Daryn Sharp added a comment -

        move command already piggybacked on copy command, so tweak it to delete the src files after it copies them

        Show
        Daryn Sharp added a comment - move command already piggybacked on copy command, so tweak it to delete the src files after it copies them
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12555649/HADOOP-9105.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1850//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1850//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12555649/HADOOP-9105.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1850//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1850//console This message is automatically generated.
        Hide
        Robert Joseph Evans added a comment -

        The changes look fine to me. I just don't totally understand why it was failing before. Is it because there is a bug in FileSystem.moveFromLocalFile? If so do we need to fix it too?

        Show
        Robert Joseph Evans added a comment - The changes look fine to me. I just don't totally understand why it was failing before. Is it because there is a bug in FileSystem.moveFromLocalFile? If so do we need to fix it too?
        Hide
        Daryn Sharp added a comment -

        Yes, I filed HADOOP-9161. This patch fixes the shell and makes it behave more consistently with other shell commands.

        Show
        Daryn Sharp added a comment - Yes, I filed HADOOP-9161 . This patch fixes the shell and makes it behave more consistently with other shell commands.
        Hide
        Robert Joseph Evans added a comment -

        OK I am +1. Even if HADOOP-9161 were in, it would not give the same detailed error messages on failure that this patch does, and it would not be as consistent with other FsShell commands. I'll check it in.

        Show
        Robert Joseph Evans added a comment - OK I am +1. Even if HADOOP-9161 were in, it would not give the same detailed error messages on failure that this patch does, and it would not be as consistent with other FsShell commands. I'll check it in.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-trunk-Commit #3146 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3146/)
        HADOOP-9105. FsShell -moveFromLocal erroneously fails (daryn via bobby) (Revision 1424566)

        Result = SUCCESS
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1424566
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Command.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/MoveCommands.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShellCopy.java
        Show
        Hudson added a comment - Integrated in Hadoop-trunk-Commit #3146 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3146/ ) HADOOP-9105 . FsShell -moveFromLocal erroneously fails (daryn via bobby) (Revision 1424566) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1424566 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Command.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/MoveCommands.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShellCopy.java
        Hide
        Robert Joseph Evans added a comment -

        Thanks Daryn,

        I put this in trunk, branch-2, and branch-0.23

        Show
        Robert Joseph Evans added a comment - Thanks Daryn, I put this in trunk, branch-2, and branch-0.23
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Yarn-trunk #72 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/72/)
        HADOOP-9105. FsShell -moveFromLocal erroneously fails (daryn via bobby) (Revision 1424566)

        Result = SUCCESS
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1424566
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Command.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/MoveCommands.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShellCopy.java
        Show
        Hudson added a comment - Integrated in Hadoop-Yarn-trunk #72 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/72/ ) HADOOP-9105 . FsShell -moveFromLocal erroneously fails (daryn via bobby) (Revision 1424566) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1424566 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Command.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/MoveCommands.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShellCopy.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #470 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/470/)
        HADOOP-9105. FsShell -moveFromLocal erroneously fails (daryn via bobby) (Revision 1424580)

        Result = UNSTABLE
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1424580
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Command.java
        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/MoveCommands.java
        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShellCopy.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #470 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/470/ ) HADOOP-9105 . FsShell -moveFromLocal erroneously fails (daryn via bobby) (Revision 1424580) Result = UNSTABLE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1424580 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Command.java /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/MoveCommands.java /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShellCopy.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1261 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1261/)
        HADOOP-9105. FsShell -moveFromLocal erroneously fails (daryn via bobby) (Revision 1424566)

        Result = FAILURE
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1424566
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Command.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/MoveCommands.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShellCopy.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1261 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1261/ ) HADOOP-9105 . FsShell -moveFromLocal erroneously fails (daryn via bobby) (Revision 1424566) Result = FAILURE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1424566 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Command.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/MoveCommands.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShellCopy.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1291 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1291/)
        HADOOP-9105. FsShell -moveFromLocal erroneously fails (daryn via bobby) (Revision 1424566)

        Result = SUCCESS
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1424566
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Command.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/MoveCommands.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShellCopy.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1291 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1291/ ) HADOOP-9105 . FsShell -moveFromLocal erroneously fails (daryn via bobby) (Revision 1424566) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1424566 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Command.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/MoveCommands.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShellCopy.java

          People

          • Assignee:
            Daryn Sharp
            Reporter:
            Daryn Sharp
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development