Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-348 Configure Samza jobs through a stream
  3. SAMZA-679

Optimize CoordinatorStream's bootstrap mechanism

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      At present, when the bootstrap using the CoordinatorStreamConsumer, we read all the messages into a set. Which is fine, if log compaction is working, but given that:
      1. The log compaction can be turned off/broken for whatever reason
      2. The is time interval between compaction

      We should consider fixing the bootstrap method to hold only the latest checkpoint (Override equals and hascode of the set is one way to go about it)

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              naveenatceg Naveen Somasundaram
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated: