[SPARK-32077] Support host-local shuffle data reading with external shuffle service disabled

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.1.0
• Fix Version/s: 3.1.0
• Component/s: Spark Core
• Assignee: wuyi
• Reporter: wuyi
• Created: 23/Jun/20 14:50
• Updated: 02/Sep/20 20:04
• Resolved: 02/Sep/20 20:04

Description

After SPARK-27651, Spark can read host-local shuffle data directly from disk with external shuffle service enabled. To extend the future, we can also support it with external shuffle service disabled.

Issue Links

• is related to: SPARK-27651 Avoid the network when block manager fetches shuffle blocks from the same host
• links to: [Github] Pull Request #28911 (Ngone51)
• links to: [Github] Pull Request #28911 (Ngone51)