Bulk Operation

SPARK-25299 Allow ShuffleDriverComponent to declare if shuffle data is reliably stored

SPARK-25299 Support json serde for the custom location

SPARK-25299 Define local/hostlocal/remote fetch for custom storage

SPARK-25299 Abstract Location in MapStatus to enable support for custom storage

SPARK-25299 Shuffle Storage API: Dynamic updates of shuffle metadata

SPARK-25299 Register shuffle map output metadata with a shuffle output tracker

SPARK-25299 Return map output metadata from shuffle writers

SPARK-25299 Use Spark plugin support to manage shuffle plugin lifecycle

SPARK-25299 Shuffle Writer API: Indeterminate shuffle support in ShuffleMapOutputWriter

SPARK-25299 Don't hold a reference to two partitionLengths arrays

SPARK-25299 Mark new Shuffle apis as @Experimental (instead of @Private)

SPARK-25299 Shuffle storage API: Use API in SortShuffleWriter

SPARK-25299 Shuffle Storage API: Use writer API in UnsafeShuffleWriter

SPARK-25299 Make Javadoc in org.apache.spark.shuffle.api visible

SPARK-25299 Shuffle Storage API: Shuffle Cleanup

SPARK-25299 Shuffle Storage API: Driver Lifecycle

SPARK-25299 Shuffle Storage API: Reads

SPARK-25299 Shuffle Storage API: Writer API and usage in BypassMergeSortShuffleWriter