Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-6100

DataNodeWebHdfsMethods does not failover in HA mode

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.4.0
    • Fix Version/s: 2.4.0
    • Component/s: ha
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      In DataNodeWebHdfsMethods, the code creates a DFSClient to connect to the NN, so that it can access the files in the cluster. DataNodeWebHdfsMethods relies on the address passed in the URL to locate the NN. This implementation has two problems:

      1. The DFSClient only knows about the current active NN, thus it does not support failover.
      2. The delegation token is based on the active NN, therefore the DFSClient will fail to authenticate of the standby NN in secure HA setup.

      Currently the parameter namenoderpcaddress in the URL stores the host-ip pair that corresponds to the active NN. To fix this bug, this jira proposes to store the name service id in the parameter in HA setup (yet the parameter stays the same in non-HA setup).

        Attachments

        1. HDFS-6100.001.patch
          37 kB
          Haohui Mai
        2. HDFS-6100.000.patch
          33 kB
          Haohui Mai

          Issue Links

            Activity

              People

              • Assignee:
                wheat9 Haohui Mai
                Reporter:
                arpitgupta Arpit Gupta
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: