Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-330

Checkpoint log and state log should resize partition count upward if necessary at job start

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 0.7.0
    • None
    • container, kv
    • None

    Description

      Currently if a the checkpoint log or state log partition count increases on job restart due to the inclusion of new streams with a higher partition count, the corresponding Kafka topics are increased to accommodate this change. In the case of the the checkpoint log, we will get a job failure at startup. We should compare the current partition size with the required and size upward if necessary at job start.

      Attachments

        Activity

          People

            Unassigned Unassigned
            jghoman Jakob Homan
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: