Uploaded image for project: 'CarbonData'
  1. CarbonData
  2. CARBONDATA-2656

Presto Stream Readers performance Enhancement

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.5.0
    • Fix Version/s: 1.5.0, 1.4.1
    • Component/s: presto-integration
    • Labels:
      None
    • Environment:
      Presto 0.187

      Description

      Background:

      In the present system, we create carbonColumnVectorImpl object in carbonVectorbatch where carbon core fill up vector data (one by one) in column matched data type array, later which at the time presto block builder call, read by stream readers (based on data type) and iterated to fill up in block and returned to presto.

      Solution:

      We can eliminate the extra iteration over the carbonColumnVectorImpl object -> vectorArray, by extending it to create a directStreamReaders which will fill up carbon-core vector data (one by one) directly to the block(presto), and on the call of block builder it will return the block to the Presto.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                sourabh.verma Sourabh Verma
                Reporter:
                sourabh.verma Sourabh Verma
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 11h 10m
                  11h 10m