[TAJO-1516] JDBC connection is lost while retrieving large results

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.10.1
• Component/s: JDBC Driver
• Assignee: Unassigned
• Reporter: Jihoon Son
• Created: 03/Apr/15 06:36
• Updated: 08/May/15 15:03
• Resolved: 08/May/15 15:03

Description

This problem is reported at Apache Tajo Korea user group (https://groups.google.com/forum/#!topic/tajo-user-kr/AjkUFEC5idg).

After a successful query execution, the JDBC connection is lost. Long time taken to retrieve data might cause the problem. The number of result was about 7 million rows.
Here are the log and network connection status.
Log

2015-03-24 22:03:06,747 ERROR: org.apache.tajo.rpc.RpcProtos (exceptionCaught(219)) - RPC Exception:요청한 주소를 배정할 수 없습니다
2015-03-24 22:03:06,748 ERROR: org.apache.tajo.rpc.NettyClientBase (operationComplete(90)) - Max retry count has been exceeded. attempts=3
2015-03-24 22:03:06,749 WARN: org.apache.tajo.client.QueryClientImpl (closeNonForwardQuery(113)) - Fail to close a TajoMaster connecti        on (qid=q_1426252658607_0120, msg=Connect error to xbdt01.mpr/10.96.16.48:26002 caused by BindException: 요청한 주소를 배정할 수 없습니다)
org.jboss.netty.channel.ConnectTimeoutException: Connect error to xbdt01.mpr/10.96.16.48:26002 caused by BindException: 요청한 주소를 배정할 수 없습니다
        at org.apache.tajo.rpc.NettyClientBase.handleConnectionInternally(NettyClientBase.java:105)
        at org.apache.tajo.rpc.NettyClientBase.connect(NettyClientBase.java:115)
        at org.apache.tajo.rpc.NettyClientBase.init(NettyClientBase.java:67)
        at org.apache.tajo.rpc.NettyClientBase.init(NettyClientBase.java:53)
        at org.apache.tajo.rpc.BlockingRpcClient.<init>(BlockingRpcClient.java:75)
        at org.apache.tajo.rpc.RpcConnectionPool.makeConnection(RpcConnectionPool.java:70)
        at org.apache.tajo.rpc.RpcConnectionPool.getConnection(RpcConnectionPool.java:85)
        at org.apache.tajo.client.SessionConnection.getTajoMasterConnection(SessionConnection.java:104)
        at org.apache.tajo.client.QueryClientImpl.closeNonForwardQuery(QueryClientImpl.java:103)
        at org.apache.tajo.jdbc.FetchResultSet.close(FetchResultSet.java:94)
        at Extract.main(Extract.java:41)

Network connection status

- Initial status
$ netstat -an | grep 26002
tcp        0      0 127.0.0.1:26002             0.0.0.0:*                   LISTEN
tcp        0      0 127.0.0.1:26002             127.0.0.1:39595             ESTABLISHED
tcp        0    116 127.0.0.1:39595             127.0.0.1:26002             ESTABLISHED

- Changed status while still retrieving results
$ netstat -an | grep 26002
tcp        0      0 127.0.0.1:26002             0.0.0.0:*                   LISTEN
tcp        0      0 127.0.0.1:39825             127.0.0.1:26002             TIME_WAIT
tcp        0      0 127.0.0.1:39632             127.0.0.1:26002             TIME_WAIT
tcp        0      0 127.0.0.1:39638             127.0.0.1:26002             TIME_WAIT
tcp        0      0 127.0.0.1:39973             127.0.0.1:26002             TIME_WAIT
tcp        0      0 127.0.0.1:39765             127.0.0.1:26002             TIME_WAIT
tcp        0      0 127.0.0.1:39944             127.0.0.1:26002             TIME_WAIT

Issue Links

• is related to: TAJO-1497 RPC client does not share a connection