[CARBONDATA-3113] Fixed Local Dictionary Query Performance and Added reusable buffer for direct flow

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.5.1
• Assignee: Kumar Vishal
• Reporter: Kumar Vishal
• Created: 21/Nov/18 07:03
• Updated: 21/Nov/18 09:11
• Resolved: 21/Nov/18 09:11

Description

1. Added reusable buffer for direct flow
In query for each page each column it is creating a byte array, when number of columns are high it is causing lots of minor gc and degrading query performance, as each page is getting uncompressed one by one we can use same buffer for all the columns and based on requested size it will resize.

2. Fixed Local Dictionary performance issue.
Reverted back #2895 and fixed NPE issue by setting null for local dictionary to vector In safe and Unsafe VariableLengthDataChunkStore

Issue Links

• links to: GitHub Pull Request #2872