Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-1516

JDBC connection is lost while retrieving large results

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.10.1
    • Component/s: JDBC Driver
    • Labels:
      None

      Description

      This problem is reported at Apache Tajo Korea user group (https://groups.google.com/forum/#!topic/tajo-user-kr/AjkUFEC5idg).

      After a successful query execution, the JDBC connection is lost. Long time taken to retrieve data might cause the problem. The number of result was about 7 million rows.
      Here are the log and network connection status.
      Log

      2015-03-24 22:03:06,747 ERROR: org.apache.tajo.rpc.RpcProtos (exceptionCaught(219)) - RPC Exception:요청한 주소를 배정할 수 없습니다
      2015-03-24 22:03:06,748 ERROR: org.apache.tajo.rpc.NettyClientBase (operationComplete(90)) - Max retry count has been exceeded. attempts=3
      2015-03-24 22:03:06,749 WARN: org.apache.tajo.client.QueryClientImpl (closeNonForwardQuery(113)) - Fail to close a TajoMaster connecti        on (qid=q_1426252658607_0120, msg=Connect error to xbdt01.mpr/10.96.16.48:26002 caused by BindException: 요청한 주소를 배정할 수 없습니다)
      org.jboss.netty.channel.ConnectTimeoutException: Connect error to xbdt01.mpr/10.96.16.48:26002 caused by BindException: 요청한 주소를 배정할 수 없습니다
              at org.apache.tajo.rpc.NettyClientBase.handleConnectionInternally(NettyClientBase.java:105)
              at org.apache.tajo.rpc.NettyClientBase.connect(NettyClientBase.java:115)
              at org.apache.tajo.rpc.NettyClientBase.init(NettyClientBase.java:67)
              at org.apache.tajo.rpc.NettyClientBase.init(NettyClientBase.java:53)
              at org.apache.tajo.rpc.BlockingRpcClient.<init>(BlockingRpcClient.java:75)
              at org.apache.tajo.rpc.RpcConnectionPool.makeConnection(RpcConnectionPool.java:70)
              at org.apache.tajo.rpc.RpcConnectionPool.getConnection(RpcConnectionPool.java:85)
              at org.apache.tajo.client.SessionConnection.getTajoMasterConnection(SessionConnection.java:104)
              at org.apache.tajo.client.QueryClientImpl.closeNonForwardQuery(QueryClientImpl.java:103)
              at org.apache.tajo.jdbc.FetchResultSet.close(FetchResultSet.java:94)
              at Extract.main(Extract.java:41)
      

      Network connection status

      - Initial status
      $ netstat -an | grep 26002
      tcp        0      0 127.0.0.1:26002             0.0.0.0:*                   LISTEN
      tcp        0      0 127.0.0.1:26002             127.0.0.1:39595             ESTABLISHED
      tcp        0    116 127.0.0.1:39595             127.0.0.1:26002             ESTABLISHED
      
      - Changed status while still retrieving results
      $ netstat -an | grep 26002
      tcp        0      0 127.0.0.1:26002             0.0.0.0:*                   LISTEN
      tcp        0      0 127.0.0.1:39825             127.0.0.1:26002             TIME_WAIT
      tcp        0      0 127.0.0.1:39632             127.0.0.1:26002             TIME_WAIT
      tcp        0      0 127.0.0.1:39638             127.0.0.1:26002             TIME_WAIT
      tcp        0      0 127.0.0.1:39973             127.0.0.1:26002             TIME_WAIT
      tcp        0      0 127.0.0.1:39765             127.0.0.1:26002             TIME_WAIT
      tcp        0      0 127.0.0.1:39944             127.0.0.1:26002             TIME_WAIT
      

        Issue Links

          Activity

          Hide
          jhkim Jinho Kim added a comment -

          It looks like problem related to ServerCallable.
          I've remove 'closeConn flag' in TAJO-1497

          Show
          jhkim Jinho Kim added a comment - It looks like problem related to ServerCallable. I've remove 'closeConn flag' in TAJO-1497
          Hide
          jihoonson Jihoon Son added a comment -

          I've tested this problem with a simple program that reads about 70GB via JDBC connection, and found it have already been solved.

          Show
          jihoonson Jihoon Son added a comment - I've tested this problem with a simple program that reads about 70GB via JDBC connection, and found it have already been solved.

            People

            • Assignee:
              Unassigned
              Reporter:
              jihoonson Jihoon Son
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development