Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-889

Change log not working properly with In memory Store

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.0
    • Fix Version/s: 0.10.1
    • Component/s: None
    • Labels:
      None

      Description

      Changle log does not work correctly with InMemory store. If you kill a job and start it again, it will not read from the beginning of the change log and bootstrap the state. Instead it will read from the newest offset.

      The reason for this bug is that the OFFSET file will be created even if the RocksDB store is not used. When a job is restated it checks if an OFFSET file exist, and if it does it will use those offsets as the offset to read from the change log. However, for an InMemory store, the state has to be rebuilt form the beginning since there is not snapshot on the disk, even though the OFFSET file is there.

      1. SAMZA-889-1.patch
        74 kB
        Yi Pan (Data Infrastructure)
      2. SAMZA-889-0.patch
        51 kB
        Navina Ramesh

        Activity

        Hide
        nickpan47 Yi Pan (Data Infrastructure) added a comment -

        Rebased, merged and submitted. Thanks!

        Show
        nickpan47 Yi Pan (Data Infrastructure) added a comment - Rebased, merged and submitted. Thanks!
        Hide
        navina Navina Ramesh added a comment -
        Show
        navina Navina Ramesh added a comment - Review Board - https://reviews.apache.org/r/46644/

          People

          • Assignee:
            navina Navina Ramesh
            Reporter:
            snoghabi Shadi A. Noghabi
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development