Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 6.0
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      This is phase 2 for adding dimensional indexing in Lucene, so we can (eventually) do efficient numeric range filtering, BigInteger/Decimal and IPv6 support, and "point in shape" spatial searching (2D or 3D).

      It's the follow-on from LUCENE-6825 (phase 1).

      This issue "just" adds DimensionalFormat (and Reader/Writer) to Codec and the IndexReader hierarchy, to IndexWriter and merging, and to document API (DimensionalField).

      I also implemented dimensional support for SimpleTextCodec, and added a test case showing that you can in fact use SimpleTextCodec to do multidimensional shape intersection (seems to pass a couple times!).

      Phase 3 will be adding support to the default codec as well ("just" wrapping BKDWriter/Reader), phase 4 is then fixing places that use the sandbox/spatial3d BKD tree to use the codec instead and maybe exposing
      sugar for numerics, things like BigInteger/Decimal, etc.

      There are many nocommits still, but I think it's close-ish ... I'll commit to a branch and iterate there.

      1. LUCENE-6852.patch
        200 kB
        Michael McCandless

        Activity

        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 1710220 from Michael McCandless in branch 'dev/branches/lucene6852'
        [ https://svn.apache.org/r1710220 ]

        LUCENE-6852: make branch

        Show
        jira-bot ASF subversion and git services added a comment - Commit 1710220 from Michael McCandless in branch 'dev/branches/lucene6852' [ https://svn.apache.org/r1710220 ] LUCENE-6852 : make branch
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 1710221 from Michael McCandless in branch 'dev/branches/lucene6852'
        [ https://svn.apache.org/r1710221 ]

        LUCENE-6852: starting patch

        Show
        jira-bot ASF subversion and git services added a comment - Commit 1710221 from Michael McCandless in branch 'dev/branches/lucene6852' [ https://svn.apache.org/r1710221 ] LUCENE-6852 : starting patch
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 1710428 from Michael McCandless in branch 'dev/branches/lucene6852'
        [ https://svn.apache.org/r1710428 ]

        LUCENE-6852: resolve nocommits, ant precommit passes

        Show
        jira-bot ASF subversion and git services added a comment - Commit 1710428 from Michael McCandless in branch 'dev/branches/lucene6852' [ https://svn.apache.org/r1710428 ] LUCENE-6852 : resolve nocommits, ant precommit passes
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 1710430 from Michael McCandless in branch 'dev/branches/lucene6852'
        [ https://svn.apache.org/r1710430 ]

        LUCENE-6852: merge trunk

        Show
        jira-bot ASF subversion and git services added a comment - Commit 1710430 from Michael McCandless in branch 'dev/branches/lucene6852' [ https://svn.apache.org/r1710430 ] LUCENE-6852 : merge trunk
        Hide
        mikemccand Michael McCandless added a comment -

        Here's an applyable patch of trunk vs the branch, I think it's ready.

        The scope on this issue is just to plumb dimensional values through document, index writer, index reader and codec APIs, but only actually implement it for SimpleTextCodec. TestDimensionalValues forces SimpleTextCodec and test various usages (including showing how BigInteger range searching will work), and seems to pass.

        Next phase is to fix the default codec to also implement it ... I'll open a follow-on issue.

        Show
        mikemccand Michael McCandless added a comment - Here's an applyable patch of trunk vs the branch, I think it's ready. The scope on this issue is just to plumb dimensional values through document, index writer, index reader and codec APIs, but only actually implement it for SimpleTextCodec. TestDimensionalValues forces SimpleTextCodec and test various usages (including showing how BigInteger range searching will work), and seems to pass. Next phase is to fix the default codec to also implement it ... I'll open a follow-on issue.
        Hide
        mikemccand Michael McCandless added a comment -

        Phase 2 is done ... next I'll make a Lucene60Codec that supports dimensional values, so it's not just simple text

        Show
        mikemccand Michael McCandless added a comment - Phase 2 is done ... next I'll make a Lucene60Codec that supports dimensional values, so it's not just simple text
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 1722882 from Michael McCandless in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1722882 ]

        LUCENE-6852: deprecate BKD tree for future dimensional values

        Show
        jira-bot ASF subversion and git services added a comment - Commit 1722882 from Michael McCandless in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1722882 ] LUCENE-6852 : deprecate BKD tree for future dimensional values

          People

          • Assignee:
            mikemccand Michael McCandless
            Reporter:
            mikemccand Michael McCandless
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development