Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-675

Check backwards incompatible config changes

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 0.10.0
    • Fix Version/s: None
    • Component/s: container
    • Labels:
      None

      Description

      One of the nice things we get for coordinator stream is, we have the previous configs we launched the job with. We can use this to our advantage for config validation. For e.g.,
      1. The number of partitions in the Kafka stream has changed, this would break our change log partition mapping.
      2. Store was opened with TTL, but reopened again without TTL in RocksDB, this can cause a corruption.

      And many more checks like this. We need a place (probably JobCoordinator) to do these validations on, validate(oldConfig from CoordinatorStream - newConfigsUsers have passed)

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              naveenatceg Naveen Somasundaram
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: