Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-2025

Ability to turn off the store for an index

Details

    • New Feature
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • None
    • 4.9, 6.0
    • core/index
    • New

    Description

      It would be really good in combination with parallel indexing if the
      Lucene store could be turned off entirely for an index.

      The reason is that part of the store is the FieldIndex (.fdx file),
      which contains an 8 bytes pointer for each document in a segment, even
      if a document does not contain any stored fields.

      With parallel indexing we will want to rewrite certain parallel
      indexes to update them, and if such an update affects only a small
      number of documents it will be a waste if you have to write the .fdx
      file every time.

      So in the case where you only want to update a data structure in the
      inverted index it makes sense to separate your index into multiple
      parallel indexes, where the ones you want to update don't contain any
      stored fields.

      It'd be also great to not only allow turning off the store but to make
      it customizable, similarly to what flexible indexing wants to achieve
      regarding the inverted index.

      As a start I'd be happy with the ability to simply turn off the store and to
      add more flexibility later.

      Attachments

        Issue Links

          Activity

            People

              michaelbusch Michael Busch
              michaelbusch Michael Busch
              Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated: