[CARBONDATA-545] Carbon Query GC Problem

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.0.0-incubating
• Component/s: data-query
• Assignee: Kumar Vishal
• Reporter: Kumar Vishal
• Due: 21/Dec/16
• Created: 20/Dec/16 06:37
• Updated: 08/Jan/17 14:31
• Resolved: 08/Jan/17 14:31

Description

Problem
There are lots of gc when carbon is processing more number of records during query, which is impacting carbon query performance.To solve this gc problem happening when query output is too huge or when more number of records are processed.

Solution
Currently we are storing all the data which is read during query from carbon data file in heap, when number of query output is huge it is causing more gc. Instead of storing in heap we can store this data in offheap and will clear when scanning is finished for that query.

Issue Links

• links to: GitHub Pull Request #450