Issue Details (XML | Word | Printable)

Key: HADOOP-2864
Type: Improvement Improvement
Status: Open Open
Priority: Major Major
Assignee: Hairong Kuang
Reporter: Hairong Kuang
Votes: 0
Watchers: 3
Operations

If you were logged in you would be able to see more operations.
Hadoop Common

Improve the Scalability and Robustness of IPC

Created: 20/Feb/08 10:46 PM   Updated: 25/Mar/08 03:03 AM
Return to search
Component/s: ipc
Affects Version/s: 0.16.0
Fix Version/s: None

Time Tracking:
Not Specified

File Attachments:
  Size
PDF File Licensed for inclusion in ASF works RPCScalabilityDesignWeb.pdf 2008-02-20 10:48 PM Hairong Kuang 85 kB
Issue Links:
Dependants
 
Incorporates
 


 Description  « Hide
This jira is intended to enhance IPC's scalability and robustness.

Currently an IPC server can easily hung due to a disk failure or garbage collection, during which it cannot respond to the clients promptly. This has caused a lot of dropped calls and delayed responses thus many running applications fail on timeout. On the other side if busy clients send a lot of requests to the server in a short period of time or too many clients communicate with the server simultaneously, the server may be swarmed by requests and cannot work responsively.

The proposed changes aim to

  1. provide a better client/server coordination
    • Server should be able to throttle client during burst of requests.
    • A slow client should not affect server from serving other clients.
    • A temporary hanging server should not cause catastrophic failures to clients.
  2. Client/server should detect remote side failures. Examples of failures include: (1) the remote host is crashed; (2) the remote host is crashed and then rebooted; (3) the remote process is crashed or shut down by an operator;
  3. Fairness. Each client should be able to make progress.


 All   Comments   Work Log   Change History   Subversion Commits      Sort Order: Ascending order - Click to sort in descending order
Hairong Kuang added a comment - 20/Feb/08 10:48 PM
Design document is attached.