Uploaded image for project: 'Apache Nemo'
  1. Apache Nemo
  2. NEMO-144 Improve Data Plane Code
  3. NEMO-148

Decouple location request from actual data transfer

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • None
    • None

    Description

      In BlockManagerWorker#readBlock, operations for (1) sending location request to the master, and (2) fetching data from a peer executor are chained with CompletedFuture#thenCompose.

        blockLocationFuture.thenCompose(responseFromMaster -> // chain

      My concern is that it is not intuitive which thread executes this piece of code, and what that implies. My guess is that

      1. Task executor thread request to the master
      2. A network thread handles reply from the master
      3. The network thread then triggers the data transfer

      We may want to decouple 2. and 3., and instead use explicit threading.

      Attachments

        Activity

          People

            Unassigned Unassigned
            johnyangk John Yang
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: