Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-8193

Refactor FailoverController/HAAdmin code to add an abstract class for "target" services

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments


    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.23.3, 0.24.0
    • 2.0.0-alpha
    • ha
    • None


      In working at HADOOP-8077, HDFS-3084, and HDFS-3072, I ran into various difficulties which are an artifact of the current design. A few of these:

      • the service name is "resolved" from the logical name (eg ns1.nn1) to an IP address at the outer layer of DFSHAAdmin
        • this means it's difficult to provide the logical name "ns1.nn1" to fence scripts (HDFS-3084)
        • this means it's difficult to configure fencing method per-namespace (since the FailoverController doesn't know what the namespace is) (HADOOP-8077)
      • the configuration for HA HDFS is weirdly split between core-site and hdfs-site, even though most users see this as an HDFS feature. For example, users expect to configure NN fencing configurations in hdfs-site, and expect the keys to have a dfs.* prefix
      • proxies are constructed at the outer layer of the admin commands. This means it's impossible for the inner layers (eg FailoverController.failover) to re-construct proxies with different timeouts (HDFS-3072)

      The proposed refactor is to add a new interface (tentatively named HAServiceTarget) which refers to target for one of the admin commands. An instance of this class is responsible for creating proxies, creating fencers, mapping back to a logical name, etc. The HDFS implementation of this class can then provide different results based on the particular nameservice, can use HDFS-specific configuration prefixes, etc. Using this class as the argument for fencing methods also makes the API more evolvable in the future, since we can add new getters to HAServiceTarget (whereas the current InetSocketAddress is quite limiting)


        1. hadoop-8193.txt
          68 kB
          Todd Lipcon
        2. hadoop-8193.txt
          68 kB
          Todd Lipcon
        3. hadoop-8193.txt
          54 kB
          Todd Lipcon
        4. hdfs-8193-fix-npe.txt
          6 kB
          Todd Lipcon

        Issue Links


          This comment will be Viewable by All Users Viewable by All Users


            tlipcon Todd Lipcon
            tlipcon Todd Lipcon
            0 Vote for this issue
            4 Start watching this issue




                Issue deployment