Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-4489

Avatica JdbcMeta statement IDs Concurrency problem

VotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Invalid
    • Affects Version/s: 1.17.0
    • Fix Version/s: 1.18.0
    • Component/s: avatica
    • Labels:
      None

      Description

        We encountered the JdbcMeta statement IDs concurrency problems that lead to errors pulling the data from avatica JDBC remote http server under heavy load.
        The error stack trace for the error is  attached.
        
        Our setup:   Avatica JDBC remote  http server handled by Calcite on the server side.
        For the record we had not been able to reproduce the same problems with Avatica, only remote proxy connected directly to DB.
       
        We believe that  the cause of the problem is unsynchronized increment of connection.statementCount
        see  MetaImpl.createStatement
      https://github.com/apache/calcite-avatica/blob/master/core/src/main/java/org/apache/calcite/avatica/MetaImpl.java#L213

      public StatementHandle createStatement(ConnectionHandle ch) {
           return new StatementHandle(ch.id, connection.statementCount++, null);
      }

       Suggested  fix is to use  AtomicInteger  for statementCount.
       Our stress tests show that the problem is resolved by the fix..
       
       

        Attachments

        1. stak-trace.txt
          2 kB
          Vlad Skarzhevskyy
        2. LoadAllData.java
          5 kB
          Vlad Skarzhevskyy
        3. stak-trace-v2.txt
          2 kB
          Vlad Skarzhevskyy

        Issue Links

          Activity

            People

            • Assignee:
              zabetak Stamatis Zampetakis
              Reporter:
              vlads Vlad Skarzhevskyy

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 20m
                20m

                  Issue deployment