Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14423

Percent (%) and plus (+) characters no longer work in WebHDFS

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.2.0, 3.1.2
    • 2.10.0, 3.3.0, 3.2.1, 3.1.3
    • webhdfs
    • None
    • Ubuntu 16.04, but I believe this is irrelevant.

    Description

      The following commands with percent (%) no longer work starting with version 3.1:

      $ hadoop/bin/hdfs dfs -touchz webhdfs://localhost/%
      $ hadoop/bin/hdfs dfs -cat webhdfs://localhost/%
      cat: URLDecoder: Incomplete trailing escape (%) pattern
      

      Also, plus (+ ) characters get turned into spaces when doing DN operations:

      $ hadoop/bin/hdfs dfs -touchz webhdfs://localhost/a+b
      $ hadoop/bin/hdfs dfs -mkdir webhdfs://localhost/c+d
      $ hadoop/bin/hdfs dfs -ls /
      Found 4 items
      -rw-r--r--   1 jing supergroup          0 2019-04-12 11:20 /a b
      drwxr-xr-x   - jing supergroup          0 2019-04-12 11:21 /c+d
      

      I can confirm that these commands work correctly on 2.9 and 3.0. Also, the usual hdfs:// client works as expected.

      I suspect a relation with HDFS-13176 or HDFS-13582, but I'm not sure what the right fix is. Note that Hive uses % to escape special characters in partition values, so banning % might not be a good option. For example, Hive will create a paths like table_name/partition_key=%2F when partition_key='/'.

      Attachments

        1. HDFS-14423.001.patch
          0.9 kB
          Wei-Chiu Chuang
        2. HDFS-14423.002.patch
          12 kB
          Masatake Iwasaki
        3. HDFS-14423.003.patch
          13 kB
          Masatake Iwasaki
        4. HDFS-14423.004.patch
          13 kB
          Masatake Iwasaki
        5. HDFS-14423-branch-2.005.patch
          13 kB
          Masatake Iwasaki
        6. HDFS-14423-branch-2.006.patch
          13 kB
          Masatake Iwasaki

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            iwasakims Masatake Iwasaki
            jingw Jing Wang
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment