Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14442

Disagreement between HAUtil.getAddressOfActive and RpcInvocationHandler.getConnectionId



    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.3.0
    • Fix Version/s: 3.3.0, 3.2.2
    • Component/s: None
    • Labels:


      While working on HDFS-14245, we noticed a discrepancy in some proxy-handling code.

      The description of RpcInvocationHandler.getConnectionId() states:

         * Returns the connection id associated with the InvocationHandler instance.
         * @return ConnectionId
        ConnectionId getConnectionId();

      It does not make any claims about whether this connection ID will be an active proxy or not. Yet in HAUtil we have:

         * Get the internet address of the currently-active NN. This should rarely be
         * used, since callers of this method who connect directly to the NN using the
         * resulting InetSocketAddress will not be able to connect to the active NN if
         * a failover were to occur after this method has been called.
         * @param fs the file system to get the active address of.
         * @return the internet address of the currently-active NN.
         * @throws IOException if an error occurs while resolving the active NN.
        public static InetSocketAddress getAddressOfActive(FileSystem fs)
            throws IOException {
          if (!(fs instanceof DistributedFileSystem)) {
            throw new IllegalArgumentException("FileSystem " + fs + " is not a DFS.");
          // force client address resolution.
          fs.exists(new Path("/"));
          DistributedFileSystem dfs = (DistributedFileSystem) fs;
          DFSClient dfsClient = dfs.getClient();
          return RPC.getServerAddress(dfsClient.getNamenode());

      Where the call RPC.getServerAddress() eventually terminates into RpcInvocationHandler#getConnectionId(), via RPC.getServerAddress() -> RPC.getConnectionIdForProxy() -> RpcInvocationHandler#getConnectionId(). HAUtil appears to be making an incorrect assumption that RpcInvocationHandler will necessarily return an active connection ID. ObserverReadProxyProvider demonstrates a counter-example to this, since the current connection ID may be pointing at, for example, an Observer NameNode.


        1. HDFS-14442.001.patch
          6 kB
          Ravuri Sushma sree
        2. HDFS-14442.002.patch
          6 kB
          Ravuri Sushma sree
        3. HDFS-14442.003.patch
          6 kB
          Ravuri Sushma sree
        4. HDFS-14442.004.patch
          5 kB
          Ravuri Sushma sree

          Issue Links



              • Assignee:
                Sushma_28 Ravuri Sushma sree
                xkrogen Erik Krogen
              • Votes:
                0 Vote for this issue
                12 Start watching this issue


                • Created: