Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14603 Über-JIRA: HDFS RBF stabilization phase II
  3. HDFS-13633

RBF: Synchronous way to create RPC client connections to NN

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: federation
    • Labels:
      None

      Description

      Currently the router code does the following.

      1. IPC handler thread gets a connection from the pool, even if the connection is NOT usable.
      2. At the same time the IPC thread also submits a request to connection creator thread for adding a new connection to the pool asynchronously.
      3. The new connection is NOT utilized by the IPC threads that get back an unusable connection.

      With this approach burst behaviors of clients, fill up the pool without necessarily using the connections. Also the approach is indeterministic.

      We propose a flag that can allow router admins to control the behavior of getting connections by the IPC handler threads. The flag would allow to toggle ON/OFF asynchronous vs synchronous way of connection creation.

      In the new model, if a connection is unusable, IPC handler thread would go ahead and create a connection and add to the pool and utilize it subsequently. It would still utilize the unusable connection if the pool is full.

       

        Attachments

          Activity

            People

            • Assignee:
              crh CR Hota
              Reporter:
              crh CR Hota
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated: