Lucene - Core
  1. Lucene - Core
  2. LUCENE-2128

Further parallelizaton of ParallelMultiSearcher

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 3.0
    • Fix Version/s: None
    • Component/s: core/search
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      When calling search(Query, Filter, int) on a ParallelMultiSearcher, the createWeights function of MultiSearcher is called, and sequentially calls docFreqs() on every sub-searcher. This can take a significant amount of time when there are lots of remote sub-searchers.

      1. LUCENE-2128.patch
        9 kB
        Simon Willnauer
      2. LUCENE-2128.patch
        9 kB
        Simon Willnauer
      3. LUCENE-2128.patch
        3 kB
        Israel Tsadok
      4. LUCENE-2128.patch
        3 kB
        Israel Tsadok

        Activity

        Hide
        Israel Tsadok added a comment -

        I extracted step 3 of createWeights into its own method, and added an overriding method on ParallelMultiSearcher.

        This is untested code, sorry.

        Show
        Israel Tsadok added a comment - I extracted step 3 of createWeights into its own method, and added an overriding method on ParallelMultiSearcher. This is untested code, sorry.
        Hide
        Simon Willnauer added a comment -

        This is untested code, sorry.

        And it does not compile either.

        Can you revise your patch?

        Show
        Simon Willnauer added a comment - This is untested code, sorry. And it does not compile either. Can you revise your patch?
        Hide
        Israel Tsadok added a comment -

        Attaching a patch that actually compiles.

        Show
        Israel Tsadok added a comment - Attaching a patch that actually compiles.
        Hide
        Simon Willnauer added a comment -

        Updated patch with cleaner structure, extracted classes and added testcase for it.

        Show
        Simon Willnauer added a comment - Updated patch with cleaner structure, extracted classes and added testcase for it.
        Hide
        Simon Willnauer added a comment -

        updated patch - createDocFrequencyMap change from protected to package private.

        Show
        Simon Willnauer added a comment - updated patch - createDocFrequencyMap change from protected to package private.
        Hide
        Uwe Schindler added a comment -

        I'll commit shortly.

        Show
        Uwe Schindler added a comment - I'll commit shortly.
        Hide
        Uwe Schindler added a comment -

        Committed revision: 888595

        Thanks Simon & Israel

        Show
        Uwe Schindler added a comment - Committed revision: 888595 Thanks Simon & Israel

          People

          • Assignee:
            Uwe Schindler
            Reporter:
            Israel Tsadok
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development