Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-2703

Reduce code generation and class loading overhead when executing queries in the EnumerableConvention

    XMLWordPrintableJSON

    Details

      Description

      The queries using Calcite's EnumerableConvention always end-up generating new java classes at runtime (using Janino) that are then instantiated using reflection. This combination of class generation and class loading introduces a big overhead in query response time.

      A quick profiling on our Company's internal test suite consisting in 4000 tests with roughly 430000 SQL queries passing through Calcite we observed that a big amount of time is spend on code generation and class loading making the EnumerableInterpretable#toBindable method a performance bottleneck. 

      Among the 430000 SQL queries there are many duplicates which are going to lead to the generation of exactly the same code Java. Introducing, a small cache at the level of EnumerableInterpretable class could avoid generating and loading the same code over and over again.

      A simple implementation based on Guava improved the overall execution time of the afforementioned test suite by more than 50%.

       

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                zabetak Stamatis Zampetakis
                Reporter:
                zabetak Stamatis Zampetakis
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 3h 10m
                  3h 10m