Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-5938

New DocIdSet implementation with random write access

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.0
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      We have a great cost API that is supposed to help make decisions about how to best execute queries. However, due to the fact that several of our filter implementations (eg. TermsFilter and BooleanFilter) return FixedBitSets, either we use the cost API and make bad decisions, or need to fall back to heuristics which are not as good such as RandomAccessFilterStrategy.useRandomAccess which decides that random access should be used if the first doc in the set is less than 100.

      On the other hand, we also have some nice compressed and cacheable DocIdSet implementation but we cannot make use of them because TermsFilter requires a DocIdSet that has random write access, and FixedBitSet is the only DocIdSet that we have that supports random access.

      I think it would be nice to replace FixedBitSet in those filters with another DocIdSet that would also support random write access but would have a better cost?

        Attachments

        1. low_freq.tasks
          0.2 kB
          Adrien Grand
        2. LUCENE-5938.patch
          74 kB
          Adrien Grand
        3. LUCENE-5938.patch
          68 kB
          Adrien Grand
        4. LUCENE-5938.patch
          70 kB
          Adrien Grand
        5. LUCENE-5938.patch
          60 kB
          Adrien Grand
        6. LUCENE-5938.patch
          12 kB
          Adrien Grand

          Activity

            People

            • Assignee:
              jpountz Adrien Grand
              Reporter:
              jpountz Adrien Grand
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: