Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-3115 Improve task broadcast latency for small tasks
  3. SPARK-3133

Piggyback get location RPC call to fetch small blocks

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Not A Problem
    • None
    • None
    • Block Manager, Spark Core
    • None

    Description

      We should add a new API to the BlockManagerMasterActor to get location or the data block directly if the data block is small.

      This effectively makes TorrentBroadcast behaves similarly to HttpBroadcast for small blocks.

      Attachments

        Activity

          People

            rxin Reynold Xin
            rxin Reynold Xin
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: