Issue Details (XML | Word | Printable)

Key: HADOOP-2188
Type: Improvement Improvement
Status: Closed Closed
Resolution: Fixed
Priority: Major Major
Assignee: Hairong Kuang
Reporter: Owen O'Malley
Votes: 0
Watchers: 2
Operations

If you were logged in you would be able to see more operations.
Hadoop Common

RPC should send a ping rather than use client timeouts

Created: 10/Nov/07 05:28 PM   Updated: 08/Jul/09 04:42 PM
Return to search
Component/s: ipc
Affects Version/s: 0.16.1
Fix Version/s: 0.18.0

Time Tracking:
Not Specified

File Attachments:
  File Name Date Attached Attached By Size
Text File Licensed for inclusion in ASF works ipc-timeout.patch 2008-02-15 07:35 PM Hairong Kuang 34 kB
Text File Licensed for inclusion in ASF works ipc-timeout1.patch 2008-03-10 10:40 PM Hairong Kuang 48 kB
Text File Licensed for inclusion in ASF works ipc-timeout2.patch 2008-03-12 12:02 AM Hairong Kuang 47 kB
Text File Licensed for inclusion in ASF works ipc-timeout3.patch 2008-03-13 08:53 PM Hairong Kuang 48 kB
Text File Licensed for inclusion in ASF works ipc-timeout4.patch 2008-03-20 11:32 PM Hairong Kuang 49 kB
Text File Licensed for inclusion in ASF works ipc-timeout5.patch 2008-03-21 06:48 PM Hairong Kuang 50 kB
Text File Licensed for inclusion in ASF works ipc-timeout6.patch 2008-04-22 11:55 PM Hairong Kuang 51 kB
Text File Licensed for inclusion in ASF works ipc-timeout7.patch 2008-04-25 08:40 PM Hairong Kuang 53 kB
Text File Licensed for inclusion in ASF works ipc-timeout8.patch 2008-04-30 04:48 PM Hairong Kuang 54 kB
Text File Licensed for inclusion in ASF works ipc-timeout9.patch 2008-05-02 11:23 PM Hairong Kuang 54 kB
Text File Licensed for inclusion in ASF works rpc-to.patch 2007-11-19 07:47 AM Owen O'Malley 27 kB
Issue Links:
Incorporates
 

Hadoop Flags: Reviewed, Incompatible change
Release Note:
Replaced timeouts with pings to check that client connection is alive. Removed the property ipc.client.timeout from the default Hadoop configuration. Removed the metric RpcOpsDiscardedOPsNum.
Resolution Date: 05/May/08 09:27 PM


 Description  « Hide
Current RPC (really IPC) relies on client side timeouts to find "dead" sockets. I propose that we have a thread that once a minute (if the connection has been idle) writes a "ping" message to the socket. The client can detect a dead socket by the resulting error on the write, so no client side timeout is required. Also note that the ipc server does not need to respond to the ping, just discard it.

 All   Comments   Work Log   Change History   Subversion Commits      Sort Order: Ascending order - Click to sort in descending order
Owen O'Malley made changes - 19/Nov/07 07:47 AM
Field Original Value New Value
Attachment rpc-to.patch [ 12369767 ]
Owen O'Malley made changes - 19/Nov/07 07:48 AM
Assignee Owen O'Malley [ owen.omalley ]
Owen O'Malley made changes - 17/Jan/08 11:27 PM
Fix Version/s 0.16.0 [ 12312740 ]
Robert Chansler made changes - 31/Jan/08 11:39 PM
Assignee Owen O'Malley [ owen.omalley ] Hairong Kuang [ hairong ]
Robert Chansler made changes - 01/Feb/08 12:35 AM
Component/s dfs [ 12310710 ]
Hairong Kuang made changes - 15/Feb/08 07:35 PM
Attachment ipc-timeout.patch [ 12375710 ]
Hairong Kuang made changes - 20/Feb/08 10:54 PM
Link This issue is part of HADOOP-2864 [ HADOOP-2864 ]
Hairong Kuang made changes - 10/Mar/08 10:40 PM
Attachment ipc-timeout1.patch [ 12377562 ]
Hairong Kuang made changes - 12/Mar/08 12:02 AM
Attachment ipc-timeout2.patch [ 12377652 ]
Hairong Kuang made changes - 13/Mar/08 08:53 PM
Attachment ipc-timeout3.patch [ 12377832 ]
Hairong Kuang made changes - 20/Mar/08 10:11 PM
Attachment ipc-timeout4.patch [ 12378353 ]
Hairong Kuang made changes - 20/Mar/08 11:30 PM
Attachment ipc-timeout4.patch [ 12378353 ]
Hairong Kuang made changes - 20/Mar/08 11:32 PM
Attachment ipc-timeout4.patch [ 12378362 ]
Hairong Kuang made changes - 20/Mar/08 11:59 PM
Fix Version/s 0.17.0 [ 12312913 ]
Affects Version/s 0.16.1 [ 12312927 ]
Status Open [ 1 ] Patch Available [ 10002 ]
Hairong Kuang made changes - 21/Mar/08 06:48 PM
Attachment ipc-timeout5.patch [ 12378398 ]
Hairong Kuang made changes - 21/Mar/08 06:48 PM
Status Patch Available [ 10002 ] Open [ 1 ]
Hairong Kuang made changes - 21/Mar/08 06:48 PM
Status Open [ 1 ] Patch Available [ 10002 ]
Robert Chansler made changes - 25/Mar/08 03:03 AM
Fix Version/s 0.17.0 [ 12312913 ]
Owen O'Malley made changes - 22/Apr/08 06:55 PM
Status Patch Available [ 10002 ] Open [ 1 ]
Hairong Kuang made changes - 22/Apr/08 11:55 PM
Attachment ipc-timeout6.patch [ 12380740 ]
Hairong Kuang made changes - 25/Apr/08 08:40 PM
Attachment ipc-timeout7.patch [ 12380967 ]
Hairong Kuang made changes - 25/Apr/08 08:41 PM
Fix Version/s 0.18.0 [ 12312972 ]
Hadoop Flags [Reviewed]
Status Open [ 1 ] Patch Available [ 10002 ]
Hairong Kuang made changes - 30/Apr/08 04:48 PM
Attachment ipc-timeout8.patch [ 12381200 ]
Hairong Kuang made changes - 30/Apr/08 04:48 PM
Status Patch Available [ 10002 ] Open [ 1 ]
Hairong Kuang made changes - 30/Apr/08 04:48 PM
Status Open [ 1 ] Patch Available [ 10002 ]
Hairong Kuang made changes - 02/May/08 11:23 PM
Status Patch Available [ 10002 ] Open [ 1 ]
Hairong Kuang made changes - 02/May/08 11:23 PM
Attachment ipc-timeout9.patch [ 12381353 ]
Hairong Kuang made changes - 02/May/08 11:27 PM
Release Note This patch introduces pings when client reads are timed out. Server does not throw away old RPCs. So if clients and the server are on different versions, they are not able to function well.
The property ipc.client.timeout is removed from the default hadoop configuration. It also removes metrics RpcOpsDiscardedOPsNum.
Hadoop Flags [Reviewed] [Incompatible change, Reviewed]
Status Open [ 1 ] Patch Available [ 10002 ]
Hairong Kuang made changes - 05/May/08 06:03 PM
Status Patch Available [ 10002 ] Open [ 1 ]
Hairong Kuang made changes - 05/May/08 06:05 PM
Hadoop Flags [Reviewed, Incompatible change] [Incompatible change, Reviewed]
Status Open [ 1 ] Patch Available [ 10002 ]
Hairong Kuang made changes - 05/May/08 09:27 PM
Resolution Fixed [ 1 ]
Hadoop Flags [Reviewed, Incompatible change] [Incompatible change, Reviewed]
Status Patch Available [ 10002 ] Resolved [ 5 ]
Robert Chansler made changes - 27/Jun/08 08:46 PM
Hadoop Flags [Reviewed, Incompatible change] [Incompatible change, Reviewed]
Release Note This patch introduces pings when client reads are timed out. Server does not throw away old RPCs. So if clients and the server are on different versions, they are not able to function well.
The property ipc.client.timeout is removed from the default hadoop configuration. It also removes metrics RpcOpsDiscardedOPsNum.
 Replaced timeouts with pings to check that client connection is alive. Removed the property ipc.client.timeout from the default Hadoop configuration. Removed the metric RpcOpsDiscardedOPsNum.
Nigel Daley made changes - 22/Aug/08 07:50 PM
Status Resolved [ 5 ] Closed [ 6 ]
Owen O'Malley made changes - 08/Jul/09 04:42 PM
Component/s dfs [ 12310710 ]

Powered by a free Atlassian JIRA open source license for Apache Software Foundation. Try JIRA - bug tracking software for your team.
Atlassian JIRA the Professional Issue Tracker. (Enterprise Edition, Version: 3.13.2-#335) - Bug/feature request - Atlassian news - Contact Administrators