[TAJO-1399] TajoResourceAllocator might hang on network error

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Fix Version/s: 0.11.0
• Component/s: RPC
• Assignee: Navis Ryu
• Reporter: Navis Ryu
• Created: 13/Mar/15 04:46
• Updated: 20/Jul/15 09:07
• Resolved: 07/May/15 05:42

Description

CallFuture<WorkerResourceAllocationResponse> callBack = new CallFuture<WorkerResourceAllocationResponse>();

...

RpcConnectionPool connPool = RpcConnectionPool.getPool();
NettyClientBase tmClient = null;
try {
  ServiceTracker serviceTracker = queryTaskContext.getQueryMasterContext().getWorkerContext().getServiceTracker();
  tmClient = connPool.getConnection(serviceTracker.getUmbilicalAddress(), QueryCoordinatorProtocol.class, true);
  QueryCoordinatorProtocolService masterClientService = tmClient.getStub();
  masterClientService.allocateWorkerResources(null, request, callBack);
} catch (Throwable e) {
  LOG.error(e.getMessage(), e);
} finally {
  connPool.releaseConnection(tmClient);
}

WorkerResourceAllocationResponse response = null;
while(!stopped.get()) {
  try {
    response = callBack.get(3, TimeUnit.SECONDS);
    ...

If "callBack" is not registered properly in netty by failed connection, etc., allocator thread would block on empty future forever, possibly making thread leakage.

Issue Links

• Is contained by: TAJO-1397 Resource allocation should be fine grained.