Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-348 Configure Samza jobs through a stream
  3. SAMZA-679

Optimize CoordinatorStream's bootstrap mechanism

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • None

    Description

      At present, when the bootstrap using the CoordinatorStreamConsumer, we read all the messages into a set. Which is fine, if log compaction is working, but given that:
      1. The log compaction can be turned off/broken for whatever reason
      2. The is time interval between compaction

      We should consider fixing the bootstrap method to hold only the latest checkpoint (Override equals and hascode of the set is one way to go about it)

      Attachments

        Activity

          People

            Unassigned Unassigned
            naveenatceg Naveen Somasundaram
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated: