-
Type:
Improvement
-
Status: Resolved
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: None
-
Fix Version/s: 1.2.0
-
Component/s: Shuffle, Spark Core
-
Labels:None
-
Target Version/s:
The attached design doc proposes a standard interface for block transferring, which will make future engineering of this functionality easier, allowing the Spark community to provide alternative implementations.
Block transferring is a critical function in Spark. All of the following depend on it:
- shuffle
- torrent broadcast
- block replication in BlockManager
- remote block reads for tasks scheduled without locality
- is related to
-
SPARK-3385 Improve shuffle performance
-
- Resolved
-
-
SPARK-3059 Spark internal module interface design
-
- Closed
-
- relates to
-
SPARK-2468 Netty-based block server / client module
-
- Resolved
-
- links to
There are no Sub-Tasks for this issue.