Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-25299 Use remote storage for persisting shuffle data
  3. SPARK-28210

Shuffle Storage API: Reads

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.1.0
    • None
    • Shuffle, Spark Core
    • None

    Description

      As part of the effort to store shuffle data in arbitrary places, this issue tracks implementing an API for reading the shuffle data stored by the write API. Also ensure that the existing shuffle implementation is refactored to use the API.

      Attachments

        Issue Links

          links to

          Web Link GitHub Pull Request #26772

          Activity

            People

              Unassigned Unassigned
              mcheah Matt Cheah
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated: