Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-6645

BKD tree queries should use BitDocIdSet.Builder

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.3, 6.0
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      When I was iterating on BKD tree originally I remember trying to use this builder (which makes a sparse bit set at first and then upgrades to dense if enough bits get set) and being disappointed with its performance.

      I wound up just making a FixedBitSet every time, but this is obviously wasteful for small queries.

      It could be the perf was poor because I was always .or'ing in DISIs that had 512 - 1024 hits each time (the size of each leaf cell in the BKD tree)? I also had to make my own DISI wrapper around each leaf cell... maybe that was the source of the slowness, not sure.

      I also sort of wondered whether the SmallDocSet in spatial module (backed by a SentinelIntSet) might be faster ... though it'd need to be sorted in the and after building before returning to Lucene.

        Attachments

        1. LUCENE-6645.patch
          13 kB
          Michael McCandless
        2. LUCENE-6645.patch
          25 kB
          Adrien Grand
        3. LUCENE-6645.patch
          51 kB
          Adrien Grand
        4. LUCENE-6645.patch
          59 kB
          Adrien Grand
        5. LUCENE-6645.patch
          3 kB
          Michael McCandless
        6. LUCENE-6645.patch
          4 kB
          Michael McCandless
        7. LUCENE-6645-spatial.patch
          10 kB
          Adrien Grand

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              mikemccand Michael McCandless
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: