Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-9148

Move the BKD index to its own file.

    XMLWordPrintableJSON

    Details

    • Type: Task
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 8.6
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      Lucene60PointsWriter stores both inner nodes and leaf nodes in the same file, interleaved. For instance if you have two fields, you would have <leaf_nodes_A, inner_nodes_A, leaf_nodes_B, inner_nodes_B>. It's not ideal since leaves and inner nodes have quite different access patterns. Should we split this into two files? In the case when the BKD index is off-heap, this would also help force it into RAM with MMapDirectory#setPreload.

      Note that Lucene60PointsFormat already has a file that it calls "index" but it's really only about mapping fields to file pointers in the other file and not what I'm discussing here. But we could possibly store the BKD indices in this existing file if we want to avoid creating a new one.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                jpountz Adrien Grand
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 10m
                  1h 10m