Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-25299 Use remote storage for persisting shuffle data
  3. SPARK-35186

Define local/hostlocal/remote fetch for custom storage

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.2.0
    • None
    • Spark Core
    • None

    Description

      After introducing the custom storage system, we need to think about how to define local/hostlocal/remote fetch for different storage systems. For example, some location implementations may not have executor ids.

      Attachments

        Issue Links

          depends upon

          Sub-task - The sub-task of the issue SPARK-34942 Abstract Location in MapStatus to enable support for custom storage

          • Major - Major loss of function.
          • In Progress

          Activity

            People

              Unassigned Unassigned
              Ngone51 wuyi
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated: