Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7586

HFTP does not work when namenode bind on wildcard

    Details

    • Type: Bug
    • Status: Reopened
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 2.2.0, 2.3.0, 2.4.0
    • Fix Version/s: None
    • Component/s: datanode
    • Labels:
      None

      Description

      When wildcard binding for NameNode RPC is turned on (i.e. dfs.namenode.rpc-address=0.0.0.0:8020), HFTP download is failing.

      Call to http://namenode:50070/data/.. returns the header Location with parameter nnaddr=0.0.0.0:8020, which is unlikely to ever succeed

      The idea would be, if wildcard binding is enabled, to get read the IP address the request is actually connected to from the HttpServletRequest and return this one.

      WDYT?

      How to reproduce:

      1. Turn on wildcard binding

      dfs.namenode.rpc-address=0.0.0.0:8020

      2. Upload a file

      $ echo "123" | hdfs dfs -put - /tmp/randomFile.txt

      3. Validate it's failing

      $ hdfs dfs -cat hftp://namenode1/tmp/randomFile.txt
      

      4. Get more details via curl

      $ curl -vv http://namenode1:50070/data/tmp/randomFile.txt?ugi=hdfs | grep "Location:"
       Location: http://datanode003:50075/streamFile/tmp/randomFile.txt?ugi=hdfs&nnaddr=0.0.0.0:8020
      

      We can clearly see the 0.0.0.0 returned as the NN ip.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              bperroud Benoit Perroud
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated: