Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-8193

Refactor FailoverController/HAAdmin code to add an abstract class for "target" services



    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.23.3, 0.24.0
    • 2.0.0-alpha
    • ha
    • None


      In working at HADOOP-8077, HDFS-3084, and HDFS-3072, I ran into various difficulties which are an artifact of the current design. A few of these:

      • the service name is "resolved" from the logical name (eg ns1.nn1) to an IP address at the outer layer of DFSHAAdmin
        • this means it's difficult to provide the logical name "ns1.nn1" to fence scripts (HDFS-3084)
        • this means it's difficult to configure fencing method per-namespace (since the FailoverController doesn't know what the namespace is) (HADOOP-8077)
      • the configuration for HA HDFS is weirdly split between core-site and hdfs-site, even though most users see this as an HDFS feature. For example, users expect to configure NN fencing configurations in hdfs-site, and expect the keys to have a dfs.* prefix
      • proxies are constructed at the outer layer of the admin commands. This means it's impossible for the inner layers (eg FailoverController.failover) to re-construct proxies with different timeouts (HDFS-3072)

      The proposed refactor is to add a new interface (tentatively named HAServiceTarget) which refers to target for one of the admin commands. An instance of this class is responsible for creating proxies, creating fencers, mapping back to a logical name, etc. The HDFS implementation of this class can then provide different results based on the particular nameservice, can use HDFS-specific configuration prefixes, etc. Using this class as the argument for fencing methods also makes the API more evolvable in the future, since we can add new getters to HAServiceTarget (whereas the current InetSocketAddress is quite limiting)


        1. hdfs-8193-fix-npe.txt
          6 kB
          Todd Lipcon
        2. hadoop-8193.txt
          68 kB
          Todd Lipcon
        3. hadoop-8193.txt
          68 kB
          Todd Lipcon
        4. hadoop-8193.txt
          54 kB
          Todd Lipcon

        Issue Links



              tlipcon Todd Lipcon
              tlipcon Todd Lipcon
              0 Vote for this issue
              4 Start watching this issue