Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-1623 High Availability Framework for HDFS NN
  3. HDFS-2713

HA : An alternative approach to clients handling Namenode failover.

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: HA branch (HDFS-1623)
    • Fix Version/s: None
    • Component/s: ha, hdfs-client
    • Labels:
      None

      Description

      This is the approach for client failover which we adopted when we developed HA for Hadoop. I would like to propose thia approach for others to review & include in the HA implementation, if found useful.

      This is similar to the ConfiguredProxyProvider in the sense that the it takes the address of both the Namenodes as the input. The major differences I can see from the current implementation are
      1) During failover, user threads can be controlled very accurately about the time they wait for active namenode to be available, awaiting the retry. Beyond this, the threads will not be made to wait; DFS Client will throw an Exception indicating that the operation has failed.
      2) Failover happens in a seperate thread, not in the client application threads. The thread will keep trying to find the Active Namenode until it succeeds.
      3) This also means that irrespective of whether the operation's RetryAction is RETRY_FAILOVER or FAIL, the user thread can trigger the client's failover.

        Attachments

          Activity

            People

            • Assignee:
              umamaheswararao Uma Maheswara Rao G
              Reporter:
              umamaheswararao Uma Maheswara Rao G
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: