Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14423

Percent (%) and plus (+) characters no longer work in WebHDFS

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.2.0, 3.1.2
    • Fix Version/s: 2.10.0, 3.3.0, 3.2.1, 3.1.3
    • Component/s: webhdfs
    • Labels:
      None
    • Environment:

      Ubuntu 16.04, but I believe this is irrelevant.

      Description

      The following commands with percent (%) no longer work starting with version 3.1:

      $ hadoop/bin/hdfs dfs -touchz webhdfs://localhost/%
      $ hadoop/bin/hdfs dfs -cat webhdfs://localhost/%
      cat: URLDecoder: Incomplete trailing escape (%) pattern
      

      Also, plus (+ ) characters get turned into spaces when doing DN operations:

      $ hadoop/bin/hdfs dfs -touchz webhdfs://localhost/a+b
      $ hadoop/bin/hdfs dfs -mkdir webhdfs://localhost/c+d
      $ hadoop/bin/hdfs dfs -ls /
      Found 4 items
      -rw-r--r--   1 jing supergroup          0 2019-04-12 11:20 /a b
      drwxr-xr-x   - jing supergroup          0 2019-04-12 11:21 /c+d
      

      I can confirm that these commands work correctly on 2.9 and 3.0. Also, the usual hdfs:// client works as expected.

      I suspect a relation with HDFS-13176 or HDFS-13582, but I'm not sure what the right fix is. Note that Hive uses % to escape special characters in partition values, so banning % might not be a good option. For example, Hive will create a paths like table_name/partition_key=%2F when partition_key='/'.

        Attachments

        1. HDFS-14423.001.patch
          0.9 kB
          Wei-Chiu Chuang
        2. HDFS-14423.002.patch
          12 kB
          Masatake Iwasaki
        3. HDFS-14423.003.patch
          13 kB
          Masatake Iwasaki
        4. HDFS-14423.004.patch
          13 kB
          Masatake Iwasaki
        5. HDFS-14423-branch-2.005.patch
          13 kB
          Masatake Iwasaki
        6. HDFS-14423-branch-2.006.patch
          13 kB
          Masatake Iwasaki

          Issue Links

            Activity

              People

              • Assignee:
                iwasakims Masatake Iwasaki
                Reporter:
                jingw Jing Wang
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: