[SPARK-6238] Support shuffle where individual blocks might be > 2G

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Component/s: Shuffle, Spark Core
• Assignee: Jin Xing
• Reporter: Reynold Xin
• Created: 09/Mar/15 23:59
• Updated: 17/May/18 21:19
• Resolved: 16/May/18 18:10

Issue Links

• duplicates: SPARK-19659 Fetch big blocks to disk when shuffle-read
• is duplicated by: SPARK-5928 Remote Shuffle Blocks cannot be more than 2 GB
• links to: [Github] Pull Request #14658 (witgo)