Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-1042

Allow offset notifications for input systems

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.12.0
    • Component/s: None
    • Labels:
      None

      Description

      Currently we keep track of the progress of each consumer using checkpointing mechanism. These checkpoints are controlled by Samza, and the system consumer is not aware of it.
      But some systems may require knowledge of the current checkpointed position in the stream.
      For this purpose we propose an optional notification mechanism for input systems.
      See the attached doc for details.

      1. SAMZA-1042-0.patch
        16 kB
        Navina Ramesh
      2. rb53453.patch
        16 kB
        Boris Shkolnik
      3. notifiable-consumer2.pdf
        34 kB
        Boris Shkolnik
      4. notifiable consumer2.md
        6 kB
        Boris Shkolnik
      5. NotifiableConsumer.pdf
        34 kB
        Boris Shkolnik
      6. NotifiableConsumer.md
        6 kB
        Boris Shkolnik

        Activity

        Show
        boryas Boris Shkolnik added a comment - RB. https://reviews.apache.org/r/53453/
        Hide
        navina Navina Ramesh added a comment -

        Attaching the diff for reference.

        Show
        navina Navina Ramesh added a comment - Attaching the diff for reference.
        Hide
        navina Navina Ramesh added a comment -

        Merged and committed.
        It will be useful to add documentation for this feature/change in contract for SystemConsumer. I will create a JIRA and assign it to you

        Show
        navina Navina Ramesh added a comment - Merged and committed. It will be useful to add documentation for this feature/change in contract for SystemConsumer. I will create a JIRA and assign it to you
        Hide
        boryas Boris Shkolnik added a comment -

        great. Thanks.

        Show
        boryas Boris Shkolnik added a comment - great. Thanks.

          People

          • Assignee:
            boryas Boris Shkolnik
            Reporter:
            boryas Boris Shkolnik
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development