Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.20.205.0, 0.22.0, 0.23.0
    • Component/s: ipc
    • Labels:
      None

      Description

      Currently Hadoop RPC does not timeout when the RPC server is alive. What it currently does is that a RPC client sends a ping to the server whenever a socket timeout happens. If the server is still alive, it continues to wait instead of throwing a SocketTimeoutException. This is to avoid a client to retry when a server is busy and thus making the server even busier. This works great if the RPC server is NameNode.

      But Hadoop RPC is also used for some of client to DataNode communications, for example, for getting a replica's length. When a client comes across a problematic DataNode, it gets stuck and can not switch to a different DataNode. In this case, it would be better that the client receives a timeout exception.

      I plan to add a new configuration ipc.client.max.pings that specifies the max number of pings that a client could try. If a response can not be received after the specified max number of pings, a SocketTimeoutException is thrown. If this configuration property is not set, a client maintains the current semantics, waiting forever.

      1. HADOOP-6889.patch
        24 kB
        John George
      2. HADOOP-6889-for20.2.patch
        28 kB
        Ravi Prakash
      3. HADOOP-6889-for20.3.patch
        28 kB
        Matt Foley
      4. HADOOP-6889-for20.patch
        24 kB
        John George
      5. HADOOP-6889-for-20security.patch
        12 kB
        John George
      6. HADOOP-6889-fortrunk.patch
        11 kB
        John George
      7. HADOOP-6889-fortrunk-2.patch
        11 kB
        John George
      8. ipcTimeout.patch
        22 kB
        Hairong Kuang
      9. ipcTimeout1.patch
        23 kB
        Hairong Kuang
      10. ipcTimeout2.patch
        23 kB
        Hairong Kuang

        Issue Links

          Activity

            People

            • Assignee:
              John George
              Reporter:
              Hairong Kuang
            • Votes:
              0 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development