Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-4329

DFSShell issues with directories with spaces in name

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0, 0.23.10, 2.1.1-beta
    • Fix Version/s: 0.23.10, 2.3.0
    • Component/s: hdfs-client
    • Labels:
      None

      Description

      This bug was discovered by Casey Ching.

      The command dfs -put /foo/hello.txt dir is supposed to create dir/hello.txt on HDFS. It doesn't work right if "dir" has a space in it:

      [adi@haus01 ~]$ hdfs dfs -mkdir 'space cat'
      [adi@haus01 ~]$ hdfs dfs -put /etc/motd 'space cat'
      [adi@haus01 ~]$ hdfs dfs -cat 'space cat/motd'
      cat: `space cat/motd': No such file or directory
      [adi@haus01 ~]$ hdfs dfs -ls space\*
      Found 1 items
      -rw-r--r--   2 adi supergroup        251 2012-12-20 11:16 space%2520cat/motd
      [adi@haus01 ~]$ hdfs dfs -cat 'space%20cat/motd'
      Welcome to Ubuntu 12.04.1 LTS (GNU/Linux 3.2.0-30-generic x86_64)
      ...
      

      Note that the dfs -ls output wrongly encodes the wrongly encoded directory name, turning %20 into %2520. It does the same thing with space:

      [adi@haus01 ~]$ hdfs dfs -touchz 'space cat/foo'
      [adi@haus01 ~]$ hdfs dfs -ls 'space cat'
      Found 1 items
      -rw-r--r--   2 adi supergroup          0 2012-12-20 11:36 space%20cat/foo
      
      1. 4329.branch-0.23.patch
        10 kB
        Cristina L. Abad
      2. 4329.trunk.patch
        10 kB
        Cristina L. Abad
      3. 4329.branch-2.patch
        9 kB
        Cristina L. Abad
      4. 4329.trunk.v2.patch
        9 kB
        Cristina L. Abad
      5. 4329.branch-0.23.v3.patch
        16 kB
        Cristina L. Abad
      6. 4329.trunk.v3.patch
        17 kB
        Cristina L. Abad

        Issue Links

          Activity

          Allen Wittenauer made changes -
          Fix Version/s 3.0.0 [ 12320356 ]
          Arun C Murthy made changes -
          Fix Version/s 2.3.0 [ 12325255 ]
          Fix Version/s 2.4.0 [ 12324588 ]
          Chuan Liu made changes -
          Link This issue is duplicated by HADOOP-10272 [ HADOOP-10272 ]
          Eli Collins made changes -
          Fix Version/s 2.1.1-beta [ 12324809 ]
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Jonathan Eagles made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s 3.0.0 [ 12320356 ]
          Fix Version/s 2.3.0 [ 12324588 ]
          Fix Version/s 0.23.10 [ 12324660 ]
          Fix Version/s 2.1.1-beta [ 12324809 ]
          Resolution Fixed [ 1 ]
          Cristina L. Abad made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Affects Version/s 0.23.10 [ 12324660 ]
          Affects Version/s 2.1.1-beta [ 12324809 ]
          Cristina L. Abad made changes -
          Attachment 4329.branch-0.23.v3.patch [ 12597759 ]
          Attachment 4329.trunk.v3.patch [ 12597760 ]
          Cristina L. Abad made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Cristina L. Abad made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Affects Version/s 3.0.0 [ 12320356 ]
          Affects Version/s 2.0.2-alpha [ 12322472 ]
          Cristina L. Abad made changes -
          Attachment 4329.trunk.v2.patch [ 12597153 ]
          Cristina L. Abad made changes -
          Attachment 4329.trunk_v2.patch [ 12597152 ]
          Cristina L. Abad made changes -
          Attachment 4329.trunk_v2.patch [ 12597152 ]
          Cristina L. Abad made changes -
          Attachment 4329.trunk.patch2 [ 12597150 ]
          Cristina L. Abad made changes -
          Attachment 4329.branch-2.patch [ 12597149 ]
          Attachment 4329.trunk.patch2 [ 12597150 ]
          Cristina L. Abad made changes -
          Attachment 4329.trunk.patch [ 12591672 ]
          Daryn Sharp made changes -
          Assignee Andy Isaacson [ adi2 ] Cristina L. Abad [ cabad ]
          Cristina L. Abad made changes -
          Attachment 4329.branch-0.23.patch [ 12590733 ]
          Andy Isaacson made changes -
          Field Original Value New Value
          Description The command {{dfs -put /foo/hello.txt dir}} is supposed to create {{dir/hello.txt}} on HDFS. It doesn't work right if "dir" has a space in it:

          {code}
          [adi@haus01 ~]$ hdfs dfs -mkdir 'space cat'
          [adi@haus01 ~]$ hdfs dfs -put /etc/motd 'space cat'
          [adi@haus01 ~]$ hdfs dfs -cat 'space cat/motd'
          cat: `space cat/motd': No such file or directory
          [adi@haus01 ~]$ hdfs dfs -ls space\*
          Found 1 items
          -rw-r--r-- 2 adi supergroup 251 2012-12-20 11:16 space%2520cat/motd
          [adi@haus01 ~]$ hdfs dfs -cat 'space%20cat/motd'
          Welcome to Ubuntu 12.04.1 LTS (GNU/Linux 3.2.0-30-generic x86_64)
          ...
          {code}

          Note that the {{dfs -ls}} output wrongly encodes the wrongly encoded directory name, turning {{%20}} into {{%2520}}. It does the same thing with space:
          {code}
          [adi@haus01 ~]$ hdfs dfs -touchz 'space cat/foo'
          [adi@haus01 ~]$ hdfs dfs -ls 'space cat'
          Found 1 items
          -rw-r--r-- 2 adi supergroup 0 2012-12-20 11:36 space%20cat/foo
          {code}
          This bug was discovered by Casey Ching.

          The command {{dfs -put /foo/hello.txt dir}} is supposed to create {{dir/hello.txt}} on HDFS. It doesn't work right if "dir" has a space in it:

          {code}
          [adi@haus01 ~]$ hdfs dfs -mkdir 'space cat'
          [adi@haus01 ~]$ hdfs dfs -put /etc/motd 'space cat'
          [adi@haus01 ~]$ hdfs dfs -cat 'space cat/motd'
          cat: `space cat/motd': No such file or directory
          [adi@haus01 ~]$ hdfs dfs -ls space\*
          Found 1 items
          -rw-r--r-- 2 adi supergroup 251 2012-12-20 11:16 space%2520cat/motd
          [adi@haus01 ~]$ hdfs dfs -cat 'space%20cat/motd'
          Welcome to Ubuntu 12.04.1 LTS (GNU/Linux 3.2.0-30-generic x86_64)
          ...
          {code}

          Note that the {{dfs -ls}} output wrongly encodes the wrongly encoded directory name, turning {{%20}} into {{%2520}}. It does the same thing with space:
          {code}
          [adi@haus01 ~]$ hdfs dfs -touchz 'space cat/foo'
          [adi@haus01 ~]$ hdfs dfs -ls 'space cat'
          Found 1 items
          -rw-r--r-- 2 adi supergroup 0 2012-12-20 11:36 space%20cat/foo
          {code}
          Andy Isaacson created issue -

            People

            • Assignee:
              Cristina L. Abad
              Reporter:
              Andy Isaacson
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development