Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7586

HFTP does not work when namenode bind on wildcard

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Reopened
    • Minor
    • Resolution: Unresolved
    • 2.2.0, 2.3.0, 2.4.0
    • None
    • datanode
    • None

    Description

      When wildcard binding for NameNode RPC is turned on (i.e. dfs.namenode.rpc-address=0.0.0.0:8020), HFTP download is failing.

      Call to http://namenode:50070/data/.. returns the header Location with parameter nnaddr=0.0.0.0:8020, which is unlikely to ever succeed

      The idea would be, if wildcard binding is enabled, to get read the IP address the request is actually connected to from the HttpServletRequest and return this one.

      WDYT?

      How to reproduce:

      1. Turn on wildcard binding

      dfs.namenode.rpc-address=0.0.0.0:8020

      2. Upload a file

      $ echo "123" | hdfs dfs -put - /tmp/randomFile.txt

      3. Validate it's failing

      $ hdfs dfs -cat hftp://namenode1/tmp/randomFile.txt
      

      4. Get more details via curl

      $ curl -vv http://namenode1:50070/data/tmp/randomFile.txt?ugi=hdfs | grep "Location:"
       Location: http://datanode003:50075/streamFile/tmp/randomFile.txt?ugi=hdfs&nnaddr=0.0.0.0:8020
      

      We can clearly see the 0.0.0.0 returned as the NN ip.

      Attachments

        1. HDFS-7586-v0.1.txt
          2 kB
          Benoit Perroud

        Activity

          People

            Unassigned Unassigned
            bperroud Benoit Perroud
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated: