Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-2578

Excessive trimming during transactional state restore

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.5.1
    • Component/s: None
    • Labels:
      None

      Description

      Samza 1.5 enabled transaction state feature by default for all samza jobs.

      We uncovered a bug related to reverting changelog state to last checkpoint (trimming), which resulted in container stuck in the restoration phase forever. This happened due to the trimming phase of state restore: when uncheckpointed messages in the changelog have their values reverted according to the job's last checkpoint. If a job needed to trim a non-zero number of messages, these trimmed messages would be repeatedly read and re-written by the restore process infinitely preventing the job from completing startup.

        Attachments

          Activity

            People

            • Assignee:
              bkonold Brett Konold
              Reporter:
              bkonold Brett Konold
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 1h 50m
                1h 50m