[CARBONDATA-830] Incorrect schedule for NewCarbonDataLoadRDD

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.0.0-incubating
• Fix Version/s: 1.1.0
• Component/s: spark-integration
• Environment: Spark 2.1 + Carbon 1.0.0
• Assignee: Weizhong
• Reporter: Weizhong
• Created: 29/Mar/17 08:25
• Updated: 30/Mar/17 05:36
• Resolved: 30/Mar/17 05:36

Description

Currently NewCarbonDataLoadRDD's getPreferredLocations will return all locs rather than 1, then on Spark may pick the same node for two tasks, so one node is getting over loaded with the task and one has no task to do, and impacting the performance despite of any failure.

Issue Links

• links to: GitHub Pull Request #708