[CARBONDATA-3426] Fix Load performance degrade by fixing task distribution

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.6.0
• Assignee: Unassigned
• Reporter: Ajantha Bhat
• Created: 11/Jun/19 11:33
• Updated: 13/Jun/19 14:30
• Resolved: 13/Jun/19 14:30

Description

Problem: Load performance degrade by fixing task distribution issue.

Cause: Consider 3 node cluster (host name a,b,c with IP1, IP2, IP3 as ip address), to launch load task, host name is required from NewCarbonDataLoadRDD in getPreferredLocations(). But if the driver is a (IP1), 

result is IP1, b,c instead of a,b,c. Hence task was not launching to one executor which is same ip as driver.

getLocalhostIPs is modified in current version recently and instead of IP it was returning address, hence local ip hostanme was removed instead of address.

solution: Revert the change in getLocalhostIPs as it is not used in any other flow.  

 

Issue Links

• links to: GitHub Pull Request #3276