[CARBONDATA-471] Optimize no kettle flow and fix issues in cluster

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.0.0-incubating
• Assignee: Ravindra Pesala
• Reporter: Ravindra Pesala
• Created: 29/Nov/16 18:01
• Updated: 02/Dec/16 08:13
• Resolved: 02/Dec/16 08:13

Description

1. In non kettle flow memory usage is very high in sort step, because it keeps lots of per every thread, it leads to GC issue.

2. Close all record readers which are opened while loading data.

3. Carbontable cannot found during data load because it is not loaded in executor, so add the carbon table in executor side to memory.

4. Prefetch the data in inputprocessor step to avoid IO waiting.

Issue Links

• links to: GitHub Pull Request #333