Uploaded image for project: 'Apache Nemo'
  1. Apache Nemo
  2. NEMO-144 Improve Data Plane Code
  3. NEMO-148

Decouple location request from actual data transfer

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:

      Description

      In BlockManagerWorker#readBlock, operations for (1) sending location request to the master, and (2) fetching data from a peer executor are chained with CompletedFuture#thenCompose.

        blockLocationFuture.thenCompose(responseFromMaster -> // chain

      My concern is that it is not intuitive which thread executes this piece of code, and what that implies. My guess is that

      1. Task executor thread request to the master
      2. A network thread handles reply from the master
      3. The network thread then triggers the data transfer

      We may want to decouple 2. and 3., and instead use explicit threading.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              johnyangk John Yang
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: