Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-67

Maintain alignment in streams when using the DefaultChooser

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 0.6.0
    • None
    • container
    • None

    Description

      The DefaultChooser currently defaults to a RoundRobinChooser when no chooser is specified in config. This means that, by default, we just round robin over which stream's messages to process next.

      It might be possible to come up with a better strategy than round robin in this scenario. This problem is outlined in detail in:

      https://wiki.apache.org/samza/Pluggable%20MessageChooser

      The most interesting and intuitive strategy seems to be to try and maintain the alignment of the streams at the time the job starts (the fourth strategy outlined in the wiki proposal).

      Also, see SAMZA-2 for more discussion on this.

      Attachments

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. SAMZA-2 Fine-grain control over stream consumption

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              Unassigned Unassigned
              criccomini Chris Riccomini
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated: