Uploaded image for project: 'REEF'
  1. REEF
  2. REEF-33

Support pushing Task messages from Evaluator to Driver.

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: REEF Driver, REEF Evaluator
    • Labels:
      None

      Description

      Currently REEF supports sending Task messages by piggybacking them to heartbeats from Evaluator to Driver. This is great for scalability, but there are cases where Task needs to send messages immediately. This question has been asked multiple times in the dev mailing list; our experience of building Surf, an in-memory caching service on REEF, confirms it.

      Although the default way of sending Task messages should use the heartbeat mechanism, we should provide an option to push a TaskMessage from Evaluator to Driver. Otherwise, we have to set up a separate channel (RemoteManager) between Evaluator and Driver, which does not make sense.

      A good approach is to reuse REEF RemoteManager between Evaluator and Driver and to provide an interface to push TaskMessages (as an option).

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                johnyangk John Yang
                Reporter:
                bgchun Byung-Gon Chun
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated: