XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.3.0, HDFS-13891
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      WebHDFS doesn't trim trailing slash causing discrepancy in operations.

      Example below
      --------------------------
      Using HDFS API, two directory are listed.

      $ hdfs dfs -ls hdfs://<router>:8888/tmp/
      Found 2 items
      drwxrwxrwx   - hdfs supergroup          0 2018-11-09 17:50 hdfs://<router>:8888/tmp/tmp1
      drwxrwxrwx   - hdfs supergroup          0 2018-11-09 17:50 hdfs://<router>:8888/tmp/tmp2
      

      Using WebHDFS API, only one directory is listed.

      $ curl -u : --negotiate -i "http://<router>:50071/webhdfs/v1/tmp/?op=LISTSTATUS"
      (snip)
      {"FileStatuses":{"FileStatus":[
      {"accessTime":0,"blockSize":0,"childrenNum":0,"fileId":16387,"group":"supergroup","length":0,"modificationTime":1552016766769,"owner":"hdfs","pathSuffix":"tmp1","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"}
      ]}}
      

      The mount table is as follows:

      $ hdfs dfsrouteradmin -ls /tmp
      Mount Table Entries:
      Source                    Destinations              Owner                     Group                     Mode                      Quota/Usage              
      /tmp                      ns1->/tmp                 aajisaka                  users                     rwxr-xr-x                 [NsQuota: -/-, SsQuota: -/-]
      /tmp/tmp1                 ns1->/tmp/tmp1            aajisaka                  users                     rwxr-xr-x                 [NsQuota: -/-, SsQuota: -/-]
      /tmp/tmp2                 ns2->/tmp/tmp2            aajisaka                  users                     rwxr-xr-x                 [NsQuota: -/-, SsQuota: -/-]
      

      Without trailing thrash, two directories are listed.

      $ curl -u : --negotiate -i "http://<router>:50071/webhdfs/v1/tmp?op=LISTSTATUS"
      (snip)
      {"FileStatuses":{"FileStatus":[
      {"accessTime":1541753421917,"blockSize":0,"childrenNum":0,"fileId":0,"group":"supergroup","length":0,"modificationTime":1541753421917,"owner":"hdfs","pathSuffix":"tmp1","permission":"777","replication":0,"storagePolicy":0,"symlink":"","type":"DIRECTORY"},
      {"accessTime":1541753429812,"blockSize":0,"childrenNum":0,"fileId":0,"group":"supergroup","length":0,"modificationTime":1541753429812,"owner":"hdfs","pathSuffix":"tmp2","permission":"777","replication":0,"storagePolicy":0,"symlink":"","type":"DIRECTORY"}
      ]}}
      

      [~ajisakaa] Thanks for reporting this, I borrowed the text from
      HDFS-13972

        Attachments

        1. HDFS-14369-HDFS-13891-regressiontest-001.patch
          2 kB
          Akira Ajisaka
        2. HDFS-14369-HDFS-13891.001.patch
          4 kB
          CR Hota
        3. HDFS-14369-HDFS-13891.002.patch
          4 kB
          CR Hota
        4. HDFS-14369-HDFS-13891.003.patch
          12 kB
          Akira Ajisaka
        5. HDFS-14369-HDFS-13891.004.patch
          13 kB
          Akira Ajisaka
        6. HDFS-14369-HDFS-13891.005.patch
          14 kB
          Akira Ajisaka
        7. HDFS-14369-HDFS-13891.006.patch
          14 kB
          Akira Ajisaka

          Issue Links

            Activity

              People

              • Assignee:
                aajisaka Akira Ajisaka
                Reporter:
                crh CR Hota
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: