Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-5015

Unexpected performance difference between SamplingAccumulator and StandardFacetAccumulator

Details

    • Bug
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 4.3
    • 4.4, 6.0
    • modules/facet
    • None
    • New, Patch Available

    Description

      I have an unexpected performance difference between the SamplingAccumulator and the StandardFacetAccumulator.

      The case is an index with about 5M documents and each document containing about 10 fields. I created a facet on each of those fields. When searching to retrieve facet-counts (using 1 CountFacetRequest), the SamplingAccumulator is about twice as fast as the StandardFacetAccumulator. This is expected and a nice speed-up.

      However, when I use more CountFacetRequests to retrieve facet-counts for more than one field, the speeds of the SampingAccumulator decreases, to the point where the StandardFacetAccumulator is faster.

       
      FacetRequests  Sampling    Standard
       1               391 ms     1100 ms
       2               531 ms     1095 ms 
       3               948 ms     1108 ms
       4              1400 ms     1110 ms
       5              1901 ms     1102 ms
      

      Is this behaviour normal? I did not expect it, as the SamplingAccumulator needs to do less work?

      Some code to show what I do:

      	searcher.search( facetsQuery, facetsCollector );
      	final List<FacetResult> collectedFacets = facetsCollector.getFacetResults();
      
      final FacetSearchParams facetSearchParams = new FacetSearchParams( facetRequests );
      
      FacetsCollector facetsCollector;
      
      if ( isSampled )
      {
      	facetsCollector =
      		FacetsCollector.create( new SamplingAccumulator( new RandomSampler(), facetSearchParams, searcher.getIndexReader(), taxo ) );
      }
      else
      {
      	facetsCollector = FacetsCollector.create( FacetsAccumulator.create( facetSearchParams, searcher.getIndexReader(), taxo ) );
      

      Attachments

        1. LUCENE-5015.patch
          18 kB
          Gilad Barkai
        2. LUCENE-5015.patch
          18 kB
          Gilad Barkai
        3. LUCENE-5015.patch
          18 kB
          Gilad Barkai
        4. LUCENE-5015.patch
          21 kB
          Gilad Barkai
        5. LUCENE-5015.patch
          19 kB
          Gilad Barkai
        6. LUCENE-5015.patch
          21 kB
          Shai Erera

        Activity

          People

            shaie Shai Erera
            robau Rob Audenaerde
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: