Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-5339

WebHDFS URI does not accept logical nameservices when security is enabled

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.2.0
    • 2.4.0
    • webhdfs
    • None
    • Reviewed

    Description

      On an insecure, HA setup, we see that this works:

      [jenkins@hdfs-cdh5-ha-1 ~]$ hdfs dfs -ls webhdfs://ns1/
      13/09/27 15:23:52 INFO web.WebHdfsFileSystem: Retrying connect to namenode: hdfs-cdh5-ha-1.ent.cloudera.com/10.20.190.104:20101. Already tried 0 time(s); retry policy is org.apache.hadoop.io.retry.RetryPolicies$FailoverOnNetworkExceptionRetry@5ebc404e, delay 0ms.
      Found 5 items
      drwxr-xr-x   - hbase hbase               0 2013-09-23 09:04 webhdfs://ns1/hbase
      drwxrwxr-x   - solr  solr                0 2013-09-18 12:07 webhdfs://ns1/solr
      drwxr-xr-x   - hdfs  supergroup          0 2013-09-19 11:09 webhdfs://ns1/system
      drwxrwxrwt   - hdfs  supergroup          0 2013-09-18 16:25 webhdfs://ns1/tmp
      drwxr-xr-x   - hdfs  supergroup          0 2013-09-18 15:53 webhdfs://ns1/user
      [jenkins@hdfs-cdh5-ha-1 ~]$
      

      However, when security is enabled, we get the following error:

      [jenkins@hdfs-cdh5-ha-secure-1 ~]$ hdfs dfs -ls webhdfs://ns1/
      -ls: java.net.UnknownHostException: ns1
      Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [<path> ...]
      [jenkins@hdfs-cdh5-ha-secure-1 ~]$
      

      I verified that we can use the hdfs://ns1/ URI on the cluster where I see the problem.

      Also, I verified on a secure, non-HA cluster that we can use the webhdfs uri in secure mode:

      [jenkins@hdfs-cdh5-secure-1 ~]$ hdfs dfs -ls webhdfs://hdfs-cdh5-secure-1.ent.cloudera.com:20101/
      drwxr-xr-x   - hbase hbase               0 2013-09-25 10:33 webhdfs://hdfs-cdh5-secure-1.ent.cloudera.com:20101/hbase
      drwxrwxr-x   - solr  solr                0 2013-09-25 10:34 webhdfs://hdfs-cdh5-secure-1.ent.cloudera.com:20101/solr
      drwxrwxrwt   - hdfs  supergroup          0 2013-09-25 10:39 webhdfs://hdfs-cdh5-secure-1.ent.cloudera.com:20101/tmp
      drwxr-xr-x   - hdfs  supergroup          0 2013-09-25 11:00 webhdfs://hdfs-cdh5-secure-1.ent.cloudera.com:20101/user
      [jenkins@hdfs-cdh5-secure-1 ~]$
      

      Attachments

        1. HDFS-5339.004.patch
          21 kB
          Haohui Mai
        2. HDFS-5339.003.patch
          21 kB
          Haohui Mai
        3. HDFS-5339.002.patch
          19 kB
          Haohui Mai
        4. HDFS-5339.001.patch
          17 kB
          Haohui Mai
        5. HDFS-5339.000.patch
          17 kB
          Haohui Mai

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            wheat9 Haohui Mai
            schu Stephen Chu
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment