Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-658

Iterator.remove breaks caching layer

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.9.0
    • Fix Version/s: 0.10.0, 0.9.1
    • Component/s: kv
    • Labels:

      Description

      If you get an iterator from the underlying layer from any store (InMemory), we just return the underlying iterator. Upon invoking remove operation on the iterator, we just delete the value from the underlying store.

      InMemory:

            override def remove(): Unit = iter.remove()
      

      This removes it from the store, but the cache still has the value and returns a value when requested for the key in the subsequent calls for the key. When the iterator.remove() is invoked, we need make sure that all the layers remove the key before we remove it from the underlying store.

      RocksDB doesn't support remove on iterator, so it's probably only for inmemory store we have this issue.

          def remove() = throw new UnsupportedOperationException("RocksDB iterator doesn't support remove")
      

        Attachments

        1. samza-658.0.9.1.patch
          19 kB
          Yi Pan (Data Infrastructure)
        2. SAMZA-658.v3.patch
          21 kB
          Guozhang Wang
        3. SAMZA-658.v4.patch
          21 kB
          Guozhang Wang

          Activity

            People

            • Assignee:
              guozhang Guozhang Wang
              Reporter:
              nsomasun Naveen
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: