Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-4142

hadoop fs -mv command creates nested directory instead of overwriting when a same named directory as source already exists

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • None
    • hadoop 0.23.4

    Description

      Using the hadoop cli when I try to move a directory to another directory, if the target directory contains a directory with the same name as the source directory, it would create nested directories instead of overwriting it. This seems counter intuitive as this is not the behavior with unix mv command.

      Here is an example to explain the bug:

      ~ $ hadoop fs -lsr /tmp/root
      lsr: DEPRECATED: Please use 'ls -R' instead.
      drwx------   - malakar hdfs          0 2012-11-01 23:30 /tmp/root/parent
      drwx------   - malakar hdfs          0 2012-11-01 23:30 /tmp/root/parent/child
      -rw-------   3 malakar hdfs       9950 2012-11-01 23:30 /tmp/root/parent/child/passwd
      drwx------   - malakar hdfs          0 2012-11-01 23:31 /tmp/root/parent2
      drwx------   - malakar hdfs          0 2012-11-01 23:31 /tmp/root/parent2/child
      ~ $ hadoop fs -mv /tmp/root/parent/child /tmp/root/parent2
      ~ $ hadoop fs -lsr /tmp/root
      lsr: DEPRECATED: Please use 'ls -R' instead.
      drwx------   - malakar hdfs          0 2012-11-01 23:32 /tmp/root/parent
      drwx------   - malakar hdfs          0 2012-11-01 23:31 /tmp/root/parent2
      drwx------   - malakar hdfs          0 2012-11-01 23:32 /tmp/root/parent2/child
      drwx------   - malakar hdfs          0 2012-11-01 23:30 /tmp/root/parent2/child/child
      -rw-------   3 malakar hdfs       9950 2012-11-01 23:30 /tmp/root/parent2/child/child/passwd
      

      The same operation seems to fail when using the FileSystem rename api though.

      Using the java api:

      ~ $ hadoop jar test.jar RenameTest
      Before:
      drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/parent
      drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/parent/child
      -rw-------   3 malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/parent/child/file
      drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/targetparent
      About to move: /tmp/renametest/parent/child to: /tmp/renametest/targetparent
      After moving: /tmp/renametest/parent/child to /tmp/renametest/targetparent
      drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/parent
      drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/targetparent
      drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/targetparent/child
      -rw-------   3 malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/targetparent/child/file
      Before:
      drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/parent
      drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/parent/child
      -rw-------   3 malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/parent/child/file
      drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/targetparent
      drwx------   - malakar hdfs          0 2012-11-02 00:23 /tmp/renametest/targetparent/child
      About to move: /tmp/renametest/parent/child to: /tmp/renametest/targetparent
      Could not rename directory: /tmp/renametest/parent/child to /tmp/renametest/targetparent
      

      Attachments

        1. RenameTest.java
          6 kB
          Arup Malakar

        Activity

          People

            Unassigned Unassigned
            amalakar Arup Malakar
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated: