Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-439

StackOverflowError in CachedStore

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.7.0
    • 0.9.0
    • kv
    • None

    Description

      When configured to large cache (100000 records) got stack overflow error on scala doublelinkedlist. It looks like reversed method is implemented very inefficiently (using recursion). The problem can be fixed by increasing stack size, but it would be nice to avoid this inefficiency.

      This could be done by replacing:

          val batch = new java.util.ArrayList[Entry[K, V]](this.dirtyCount)
          for (k <- this.dirty.reverse) {
            val entry = this.cache.get(k)
            entry.dirty = null // not dirty any more
            batch.add(new Entry(k, entry.value))
          }
          store.putAll(batch)
      

      with:

          val batch = new Array[Entry[K, V]](this.dirtyCount)
          var pos : Int = this.dirtyCount - 1;
          for (k <- this.dirty) {
            val entry = this.cache.get(k)
            entry.dirty = null // not dirty any more
            batch(pos) = new Entry(k, entry.value)
            pos -= 1
          }
          store.putAll(util.Arrays.asList(batch : _*))
      

      Attachments

        1. SAMZA-439.patch
          3 kB
          Manikumar

        Activity

          People

            omkreddy Manikumar
            dmitrybugaychenko Dmitry Bugaychenko
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: