Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-658

Iterator.remove breaks caching layer

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.9.0
    • 0.10.0, 0.9.1
    • kv

    Description

      If you get an iterator from the underlying layer from any store (InMemory), we just return the underlying iterator. Upon invoking remove operation on the iterator, we just delete the value from the underlying store.

      InMemory:

            override def remove(): Unit = iter.remove()
      

      This removes it from the store, but the cache still has the value and returns a value when requested for the key in the subsequent calls for the key. When the iterator.remove() is invoked, we need make sure that all the layers remove the key before we remove it from the underlying store.

      RocksDB doesn't support remove on iterator, so it's probably only for inmemory store we have this issue.

          def remove() = throw new UnsupportedOperationException("RocksDB iterator doesn't support remove")
      

      Attachments

        1. samza-658.0.9.1.patch
          19 kB
          Yi Pan
        2. SAMZA-658.v3.patch
          21 kB
          Guozhang Wang
        3. SAMZA-658.v4.patch
          21 kB
          Guozhang Wang

        Activity

          People

            guozhang Guozhang Wang
            nsomasun Naveen
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: