Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-2578

Excessive trimming during transactional state restore

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 1.5.1
    • None
    • None

    Description

      Samza 1.5 enabled transaction state feature by default for all samza jobs.

      We uncovered a bug related to reverting changelog state to last checkpoint (trimming), which resulted in container stuck in the restoration phase forever. This happened due to the trimming phase of state restore: when uncheckpointed messages in the changelog have their values reverted according to the job's last checkpoint. If a job needed to trim a non-zero number of messages, these trimmed messages would be repeatedly read and re-written by the restore process infinitely preventing the job from completing startup.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            bkonold Brett Konold
            bkonold Brett Konold
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 1h 50m
                1h 50m

                Slack

                  Issue deployment