Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-8169

[Java] Improve the performance of JDBC adapter by allocating memory proactively

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.0.0
    • Java

    Description

      The current implementation use setSafe methods to dynamically allocate memory if necessary. For fixed width vectors (which are frequently used in JDBC), however, we can allocate memory proactively, since the vector size is known as a configuration parameter. So for fixed width vectors, we can use set methods instead.

      This change leads to two benefits:
      1. When processing each value, we no longer have to check vector capacity and reallocate memroy if needed. This leads to better performance.
      2. If we allow the memory to expand automatically (each time by 2x), the amount of memory usually ends up being more than necessary. By allocating memory by the configuration parameter, we allocate no more, or no less.

      Benchmark results show notable performance improvements:

      Before:

      Benchmark Mode Cnt Score Error Units
      JdbcAdapterBenchmarks.consumeBenchmark avgt 5 521.700 ± 4.837 us/op

      After:

      Benchmark Mode Cnt Score Error Units
      JdbcAdapterBenchmarks.consumeBenchmark avgt 5 430.523 ± 9.932 us/op

      Attachments

        Issue Links

          Activity

            People

              fan_li_ya Liya Fan
              fan_li_ya Liya Fan
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 20m
                  1h 20m