Apache S4
  1. Apache S4
  2. S4-21

Create standard PEs, and Event classes

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.5.0
    • Fix Version/s: 0.5.0
    • Labels:
      None

      Description

      We should provide some reusable classes such as JoinPE and a MapEvent. The MapEvent is a schema-less reusable event class.

        Activity

        Hide
        Leo Neumeyer added a comment -

        What do you think of the following API for MapEvent:

        package org.apache.s4.base;

        public class MapEvent extends Event {

        public <T> void put(Class<T> type, String name, T value) {

        }

        public <T> T get(String name)

        { T x = null; // get object. // throw exception if type doesn't match. return x; }

        }

        Show
        Leo Neumeyer added a comment - What do you think of the following API for MapEvent: package org.apache.s4.base; public class MapEvent extends Event { public <T> void put(Class<T> type, String name, T value) { } public <T> T get(String name) { T x = null; // get object. // throw exception if type doesn't match. return x; } }
        Hide
        J Mohamed Zahoor added a comment -

        A generic stream (default stream) class to carry this MapEvent may be desirable in some scenarios.
        A user can just create PEs (which uses MapEvent)

        Also how about giving some generic PEs ....like PEs which can do math/string/date operations...

        Show
        J Mohamed Zahoor added a comment - A generic stream (default stream) class to carry this MapEvent may be desirable in some scenarios. A user can just create PEs (which uses MapEvent) Also how about giving some generic PEs ....like PEs which can do math/string/date operations...
        Hide
        Leo Neumeyer added a comment -

        Makes sense to have a generic stream, I think.

        We could have a factory method for the generic stream that only takes these arguments:

        • stream name
        • key finder class or key finder string
        • target PE

        (finder string is in the todo list, we already have it in 0.4. the string is parsed to generate the finder class. It's an option for developers who dont' want to write key finder classes.)

        Another thought is, should the Event base class be the generic event class? If the map is not created the overhead would be limited to one reference field. The benefit is that all events will be augmentable without having to subclass.

        Why would we do this? because it provides an easy way to add meta-data or annotations to events. For example, for 2-way communication we need to send the origin address in the event over several hops. In NLP, we may want to add annotations to the event as it traverses several PEs. The annotator chain may change often, making it impractical to use dedicated event classes. The Map in base Event will make it possible to append these type of info using a simple pattern. (Of course this is more costly than using event classes but requires less boilerplate and fewer classes.) This design provide flexibility to use classes for static typing and speed or maps for flexibility or a combination of both.

        Show
        Leo Neumeyer added a comment - Makes sense to have a generic stream, I think. We could have a factory method for the generic stream that only takes these arguments: stream name key finder class or key finder string target PE (finder string is in the todo list, we already have it in 0.4. the string is parsed to generate the finder class. It's an option for developers who dont' want to write key finder classes.) Another thought is, should the Event base class be the generic event class? If the map is not created the overhead would be limited to one reference field. The benefit is that all events will be augmentable without having to subclass. Why would we do this? because it provides an easy way to add meta-data or annotations to events. For example, for 2-way communication we need to send the origin address in the event over several hops. In NLP, we may want to add annotations to the event as it traverses several PEs. The annotator chain may change often, making it impractical to use dedicated event classes. The Map in base Event will make it possible to append these type of info using a simple pattern. (Of course this is more costly than using event classes but requires less boilerplate and fewer classes.) This design provide flexibility to use classes for static typing and speed or maps for flexibility or a combination of both.
        Show
        Leo Neumeyer added a comment - We can now dynamically add attributes to Event in a type safe manner. Please review the code: https://github.com/leoneu/s4-piper/blob/master/subprojects/s4-base/src/main/java/org/apache/s4/base/Event.java Here is test case: https://github.com/leoneu/s4-piper/blob/master/subprojects/s4-core/src/test/java/org/apache/s4/base/EventAttributeTest.java Please review.
        Hide
        Matthieu Morel added a comment -

        This was committed in e228a8aeb423feba5d58a24fb160c63c6ac8efca , merged into piper and it is working fine

        Show
        Matthieu Morel added a comment - This was committed in e228a8aeb423feba5d58a24fb160c63c6ac8efca , merged into piper and it is working fine

          People

          • Assignee:
            Leo Neumeyer
            Reporter:
            Leo Neumeyer
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development