Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-2124 Namenode HA using Backup Namenode as Hot Standby
  3. HDFS-2165

The Intelligent Clients should switch dynamically according to the Namenode switchover & the operations that fail during the switching should be retried on the newly Active Namenode.

    Details

    • Type: Sub-task Sub-task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: datanode, hdfs-client, namenode
    • Labels:
      None

      Description

      Here we consider DataNode and DFSClient as clients to the Namenode. When Active Namenode goes down and Standby takes over the role, the clients should understand it and communicate to the newly Active Namenode from that point of time.

        Activity

        Hide
        Justin Joseph added a comment -

        The high level approach is mentioned below.
        a) The client will have a list of IP addresses of the Active and Standby, which it uses for the connection string.
        b) For connecting to Namenode for the first time, the client will try to ping both the Namenodes. Whoever is reachable, client will treat him as the Active Namenode.
        c) The information on which Namenode is Active is cached. For further client requests, DFSClient will route them to the Namenode whose URL is cached. This avoids the overhead of finding the Namenode always through trial and error.
        d) When the client detects Active Namenode failure, it will start the switching procedure. Client keeps trying to ping both the Namenodes configured, until it can successfully connect to any one of them.

        Show
        Justin Joseph added a comment - The high level approach is mentioned below. a) The client will have a list of IP addresses of the Active and Standby, which it uses for the connection string. b) For connecting to Namenode for the first time, the client will try to ping both the Namenodes. Whoever is reachable, client will treat him as the Active Namenode. c) The information on which Namenode is Active is cached. For further client requests, DFSClient will route them to the Namenode whose URL is cached. This avoids the overhead of finding the Namenode always through trial and error. d) When the client detects Active Namenode failure, it will start the switching procedure. Client keeps trying to ping both the Namenodes configured, until it can successfully connect to any one of them.
        Hide
        Suresh Srinivas added a comment -

        Justin, have you looked at subtasks under HDFS-1623. Could we avoid creating duplicate jiras?

        Show
        Suresh Srinivas added a comment - Justin, have you looked at subtasks under HDFS-1623 . Could we avoid creating duplicate jiras?
        Hide
        Uma Maheswara Rao G added a comment -

        HDFS-1973 implemented such mechanism..

        Show
        Uma Maheswara Rao G added a comment - HDFS-1973 implemented such mechanism..

          People

          • Assignee:
            Unassigned
            Reporter:
            Justin Joseph
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development