Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-4942

Indexed non-point shapes index excessive terms

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 5.1
    • modules/spatial
    • None
    • New

    Description

      Indexed non-point shapes are comprised of a set of terms that represent grid cells. Cells completely within the shape or cells on the intersecting edge that are at the maximum detail depth being indexed for the shape are denoted as "leaf" cells. Such cells have a trailing '+' at the end. Such tokens are actually indexed twice, one with the leaf byte and one without.

      The TermQuery based PrefixTree Strategy doesn't consider the notion of 'leaf' cells and so the tokens with '+' are completely redundant.

      The Recursive [algorithm] based PrefixTree Strategy better supports correct search of indexed non-point shapes than TermQuery does and the distinction is relevant. However, the foundational search algorithms used by this strategy (Intersects & Contains; the other 2 are based on these) could each be upgraded to deal with this correctly. Not trivial but very doable.

      In the end, spatial non-point indexes can probably be trimmed my ~40% by doing this.

      Attachments

        1. LUCENE-4942_non-point_excessive_terms.patch
          68 kB
          David Smiley
        2. LUCENE-4942_non-point_excessive_terms.patch
          61 kB
          David Smiley
        3. LUCENE-4942-clone.diff
          3 kB
          Ryan McKinley
        4. spatial.alg
          4 kB
          David Smiley

        Issue Links

          Activity

            People

              dsmiley David Smiley
              dsmiley David Smiley
              Votes:
              1 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Slack

                  Issue deployment