Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-5339

WebHDFS URI does not accept logical nameservices when security is enabled

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.2.0
    • Fix Version/s: 2.4.0
    • Component/s: webhdfs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      On an insecure, HA setup, we see that this works:

      [jenkins@hdfs-cdh5-ha-1 ~]$ hdfs dfs -ls webhdfs://ns1/
      13/09/27 15:23:52 INFO web.WebHdfsFileSystem: Retrying connect to namenode: hdfs-cdh5-ha-1.ent.cloudera.com/10.20.190.104:20101. Already tried 0 time(s); retry policy is org.apache.hadoop.io.retry.RetryPolicies$FailoverOnNetworkExceptionRetry@5ebc404e, delay 0ms.
      Found 5 items
      drwxr-xr-x   - hbase hbase               0 2013-09-23 09:04 webhdfs://ns1/hbase
      drwxrwxr-x   - solr  solr                0 2013-09-18 12:07 webhdfs://ns1/solr
      drwxr-xr-x   - hdfs  supergroup          0 2013-09-19 11:09 webhdfs://ns1/system
      drwxrwxrwt   - hdfs  supergroup          0 2013-09-18 16:25 webhdfs://ns1/tmp
      drwxr-xr-x   - hdfs  supergroup          0 2013-09-18 15:53 webhdfs://ns1/user
      [jenkins@hdfs-cdh5-ha-1 ~]$
      

      However, when security is enabled, we get the following error:

      [jenkins@hdfs-cdh5-ha-secure-1 ~]$ hdfs dfs -ls webhdfs://ns1/
      -ls: java.net.UnknownHostException: ns1
      Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [<path> ...]
      [jenkins@hdfs-cdh5-ha-secure-1 ~]$
      

      I verified that we can use the hdfs://ns1/ URI on the cluster where I see the problem.

      Also, I verified on a secure, non-HA cluster that we can use the webhdfs uri in secure mode:

      [jenkins@hdfs-cdh5-secure-1 ~]$ hdfs dfs -ls webhdfs://hdfs-cdh5-secure-1.ent.cloudera.com:20101/
      drwxr-xr-x   - hbase hbase               0 2013-09-25 10:33 webhdfs://hdfs-cdh5-secure-1.ent.cloudera.com:20101/hbase
      drwxrwxr-x   - solr  solr                0 2013-09-25 10:34 webhdfs://hdfs-cdh5-secure-1.ent.cloudera.com:20101/solr
      drwxrwxrwt   - hdfs  supergroup          0 2013-09-25 10:39 webhdfs://hdfs-cdh5-secure-1.ent.cloudera.com:20101/tmp
      drwxr-xr-x   - hdfs  supergroup          0 2013-09-25 11:00 webhdfs://hdfs-cdh5-secure-1.ent.cloudera.com:20101/user
      [jenkins@hdfs-cdh5-secure-1 ~]$
      

        Attachments

        1. HDFS-5339.000.patch
          17 kB
          Haohui Mai
        2. HDFS-5339.001.patch
          17 kB
          Haohui Mai
        3. HDFS-5339.002.patch
          19 kB
          Haohui Mai
        4. HDFS-5339.003.patch
          21 kB
          Haohui Mai
        5. HDFS-5339.004.patch
          21 kB
          Haohui Mai

          Activity

            People

            • Assignee:
              wheat9 Haohui Mai
              Reporter:
              schu Stephen Chu
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: