Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-8562

Speed up merging segments of points with data dimensions

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 7.7, 8.0
    • 7.7, 8.0
    • core/index
    • None
    • New

    Description

      Currently when merging segments of points with data dimensions, all dimensions are sorted and carried over down the tree even though only indexing dimensions are needed to build the BKD tree. This is needed so leaf node data can be compressed by common prefix.

      But when using MutablePointValues, this ordering is done at the leaf level so we can se a similar approach from data dimensions and delay the sorting at leaf level. This seems to speed up indexing time as well as reduce the storage needed for building the index.

       

       

       

      Attachments

        1. LUCENE-8562.patch
          10 kB
          Ignacio Vera
        2. LUCENE-8562.patch
          8 kB
          Ignacio Vera
        3. LUCENE-8562.patch
          8 kB
          Ignacio Vera
        4. LUCENE-8562.patch
          9 kB
          Ignacio Vera
        5. LUCENE-8562.patch
          11 kB
          Ignacio Vera
        6. LUCENE-8562.patch
          11 kB
          Ignacio Vera
        7. LUCENE-8562.patch
          10 kB
          Ignacio Vera

        Activity

          People

            ivera Ignacio Vera
            ivera Ignacio Vera
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: