Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.6, 4.0-ALPHA
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New, Patch Available

      Description

      I've noticed that when doing thousands of searches in a single thread the average time is quite low i.e. a few milliseconds. When adding more concurrent searches doing exactly the same search the average time increases drastically.
      I've profiled the search classes and found that the whole of lucene blocks on

      org.apache.lucene.index.SegmentCoreReaders.getTermsReader
      org.apache.lucene.util.VirtualMethod
      public synchronized int getImplementationDistance
      org.apache.lucene.util.AttributeSourcew.getAttributeInterfaces

      These cause search times to increase from a few milliseconds to up to 2 seconds when doing 500 concurrent searches on the same in memory index. Note: That the index is not being updates at all, so not refresh methods are called at any stage.

      Some questions:
      Why do we need synchronization here?
      There must be a non-lockable solution for these, they basically cause lucene to be ok for single thread applications but disastrous for any concurrent implementation.

      I'll do some experiments by removing the synchronization from the methods of these classes.

      1. App.java
        3 kB
        Gerrit Jansen van Vuuren
      2. LUCENE-3653.patch
        0.9 kB
        Simon Willnauer
      3. LUCENE-3653.patch
        2 kB
        Simon Willnauer
      4. LUCENE-3653.patch-BiasedLockingStartupDelay_1.png
        130 kB
        Gerrit Jansen van Vuuren
      5. LUCENE-3653.patch-BiasedLockingStartupDelay_2.png
        129 kB
        Gerrit Jansen van Vuuren
      6. LUCENE-3653.patch-BiasedLockingStartupDelay_3.png
        129 kB
        Gerrit Jansen van Vuuren
      7. LUCENE-3653-no-sync.png
        33 kB
        Simon Willnauer
      8. LUCENE-3653-sync-.png
        32 kB
        Simon Willnauer
      9. LUCENE-3653-VirtualMethod+AttributeSource.patch
        21 kB
        Uwe Schindler
      10. LUCENE-3653-VirtualMethod+AttributeSource.patch
        17 kB
        Uwe Schindler
      11. LUCENE-3653-VirtualMethod+AttributeSource.patch
        17 kB
        Uwe Schindler
      12. lucene-unsync.diff
        17 kB
        Gerrit Jansen van Vuuren
      13. profile_1_a.png
        133 kB
        Gerrit Jansen van Vuuren
      14. profile_1_b.png
        143 kB
        Gerrit Jansen van Vuuren
      15. profile_1_c.png
        162 kB
        Gerrit Jansen van Vuuren
      16. profile_1_d.png
        171 kB
        Gerrit Jansen van Vuuren
      17. profile_2_a.png
        128 kB
        Gerrit Jansen van Vuuren
      18. profile_2_b.png
        160 kB
        Gerrit Jansen van Vuuren
      19. profile_2_c.png
        135 kB
        Gerrit Jansen van Vuuren
      20. Threads-LUCENE-3653.patch.png
        128 kB
        Gerrit Jansen van Vuuren

        Activity

          People

          • Assignee:
            Simon Willnauer
            Reporter:
            Gerrit Jansen van Vuuren
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development