[CARBONDATA-2656] Presto Stream Readers performance Enhancement

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.5.0
• Fix Version/s: 1.5.0, 1.4.1
• Component/s: presto-integration
• Environment: Presto 0.187
• Assignee: Sourabh Verma
• Reporter: Sourabh Verma
• Created: 26/Jun/18 10:25
• Updated: 18/Jul/18 17:34
• Resolved: 18/Jul/18 06:57

Description

Background:

In the present system, we create carbonColumnVectorImpl object in carbonVectorbatch where carbon core fill up vector data (one by one) in column matched data type array, later which at the time presto block builder call, read by stream readers (based on data type) and iterated to fill up in block and returned to presto.

Solution:

We can eliminate the extra iteration over the carbonColumnVectorImpl object -> vectorArray, by extending it to create a directStreamReaders which will fill up carbon-core vector data (one by one) directly to the block(presto), and on the call of block builder it will return the block to the Presto.

Issue Links

• links to: GitHub Pull Request #2412