Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-4163

Improve concurrency in MMapIndexInput.clone()

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.6, 4.0-ALPHA, 6.0
    • Fix Version/s: 4.0-ALPHA, 3.6.1, 6.0
    • Component/s: core/store
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      Followup issue from SOLR-3566:

      Whenever you clone the TermIndex, it also creates a clone of the underlying IndexInputs. In high cocurrent environments, the clone method of MMapIndexInput is a bottleneck (it has heavy work to do to manage the weak references in a synchronized block).

      Everywhere else in Lucene we use my new WeakIdentityMap for managing concurrent weak maps. For this case I did not do this, as the WeakIdentityMap has no iterators (it doe snot implement Map interface). This issue will add a key and values iterator (the key iterator will not return GC'ed keys), so MMapIndexInput can use WeakIdentityMap backed by ConcurrentHashMap and needs no synchronization. ConcurrentHashMap has better concurrency because it distributes the hash keys in different buckets per thread.

        Attachments

        1. LUCENE-4163.patch
          9 kB
          Uwe Schindler
        2. LUCENE-4163.patch
          10 kB
          Uwe Schindler
        3. LUCENE-4163.patch
          11 kB
          Uwe Schindler
        4. LUCENE-4163-3.x
          12 kB
          Uwe Schindler

          Issue Links

            Activity

              People

              • Assignee:
                thetaphi Uwe Schindler
                Reporter:
                thetaphi Uwe Schindler
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: