Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.5.0
    • Fix Version/s: 0.5.0
    • Labels:
      None

      Description

      Need an adaptor for v0.5

      Idea I posted earlier:

      What do you think of this idea for a simple adaptor:

      • Adaptor extends App
      • Adaptor can send events but not receive (for now)
      • Adaptor is deployed as a regular App to the S4 cluster and as an
        Adaptor type in a host (separate from the S4 cluster).
      • Adaptor, unlike regular apps, can accept event data (in any format)
        directly, not via comm layer.
      • Input data is transformed into S4 events using a modular approach
        and by providing standard modules such as JSON.
      • Output events are exposed using EventSource and consumed by other
        apps without even knowing that they are Adaptors (only the App type is
        exposed in the cluster).
      • S4 events can be processed locally using PEs and Streams as usual.
        (We kind of need to get a local Sender for the local PEs and a
        standard cluster Sender for the EventSource object.)

      So why this approach?

      The GOOD:

      • Seems to be the least disruptive way to inject external events
      • Apps can easily consume the events in a modular way without any
        dependencies. Getting events from an adaptor or from another app is
        identical.
      • The adaptor would be packaged and deployed to the cluster as if it
        was an App (no incremental cost)
      • The adaptor can do preprocessing using the same programming model
        and can reuse PEs.

      The CHALLENGE:

      • We need to also deploy the Adaptor in a separate host. On the other
        hand, this is inevitable. At least we use the same approach instead of
        creating a different system.
      • The Adaptor will need to be integrated with ZK to get the physical addresses.
      • We need to deal with two senders.

      for later: two-way communication and adapter clusters.

      thoughts?

      1. s4-subclusters.pdf
        930 kB
        Leo Neumeyer
      2. Inter cluster communication in S4 piper.pdf
        1.10 MB
        Matthieu Morel
      There are no Sub-Tasks for this issue.

        Activity

          People

          • Assignee:
            Matthieu Morel
            Reporter:
            Leo Neumeyer
          • Votes:
            1 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development