[TAJO-1540] RpcCallback must be able to handle TimeoutException or cancel.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.11.0
• Assignee: Unassigned
• Reporter: Hyoungjun Kim
• Created: 08/Apr/15 02:41
• Updated: 02/Sep/15 05:23
• Resolved: 02/Sep/15 05:23

Description

I investigated the lock of CallFuture while reviewing TAJO-1469. CallFuture should be synchronized with run() and get(). Current code looks like this would be implemented but not. If the following situation is occur, some resources or tasks will be lost forever.

Worker: TaskRunner sends GetTask request.
QM: QueryMaster selects proper task and calls RpcCallback.
Worker: AsyncRpcClient receives the response and calls CallFuture.run(response). 3-1. Worker: If TimeoutException occurs after 1) between 2) ~ 3), TaskRunner can't receive the response and doesn't run the allocated task, but QM doesn't know about that.

We should fix this problem in the RPC module and add a right cancel logic.

Issue Links

• Is contained by: TAJO-1563 Improve RPC error handling
• is related to: TAJO-1469 allocateQueryMaster can leak resources if it times-out (3sec, hardcoded)