Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-8982

Make NativeUnixDirectory pure java now that direct IO is possible

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: main (9.0)
    • Component/s: modules/misc
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      NativeUnixDirectory is a Directory implementation that uses direct IO to write newly merged segments.  Direct IO bypasses the kernel's buffer cache and write cache, making merge writes "invisible" to the kernel, though the reads for merging the N segments are still going through the kernel.

      But today, NativeUnixDirectory uses a small JNI wrapper to access the O_DIRECT flag to open ... since JDK9 we can now pass that flag in pure java code, so we should now fix NativeUnixDirectory to not use JNI anymore.

      We should also run some more realistic benchmarks seeing if this option really helps nodes that are doing concurrent indexing (merging) and searching.

        Attachments

          Activity

            People

            • Assignee:
              uschindler Uwe Schindler
              Reporter:
              mikemccand Michael McCandless
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 31h 50m
                31h 50m