Lucene - Core
  1. Lucene - Core
  2. LUCENE-5724

CompoundFileWriter loses the IOContext sometimes

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.9, 6.0
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      Nightly build hit OOME with this

      ant test  -Dtestcase=Test2BPostings -Dtests.method=test -Dtests.seed=33378E77AE43B10B -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/hudson/lucene-data/enwiki.random.lines.txt -Dtests.locale=sl -Dtests.timezone=America/Argentina/ComodRivadavia -Dtests.file.encoding=UTF-8
      

      The test was using NRTCachingDirectory, but the OOME happens because CompoundFileWriter's getOutput fails to pass down the incoming IOContext.

      IndexWriter has properly set up the IOContext for flush, put a huge file size in there, but by the time NRTCachingDirectory saw it, it was 0 bytes, and then many 100s of MB proceeded to be written into the RAMFile.

      1. LUCENE-5724.patch
        3 kB
        Michael McCandless

        Activity

        Hide
        Michael McCandless added a comment -

        Patch (for 4.x), I think it's ready. It fixes the build failure, and I added a specific test case when creating a large CFS using NRTCachingDirectory.

        Show
        Michael McCandless added a comment - Patch (for 4.x), I think it's ready. It fixes the build failure, and I added a specific test case when creating a large CFS using NRTCachingDirectory.
        Hide
        ASF subversion and git services added a comment -

        Commit 1599288 from Michael McCandless in branch 'dev/branches/branch_4x'
        [ https://svn.apache.org/r1599288 ]

        LUCENE-5724: fix CompoundFileWriter to not suppress the incoming IOContext

        Show
        ASF subversion and git services added a comment - Commit 1599288 from Michael McCandless in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1599288 ] LUCENE-5724 : fix CompoundFileWriter to not suppress the incoming IOContext
        Hide
        ASF subversion and git services added a comment -

        Commit 1599291 from Michael McCandless in branch 'dev/trunk'
        [ https://svn.apache.org/r1599291 ]

        LUCENE-5724: fix CompoundFileWriter to not suppress the incoming IOContext

        Show
        ASF subversion and git services added a comment - Commit 1599291 from Michael McCandless in branch 'dev/trunk' [ https://svn.apache.org/r1599291 ] LUCENE-5724 : fix CompoundFileWriter to not suppress the incoming IOContext
        Hide
        ASF subversion and git services added a comment -

        Commit 1599292 from Michael McCandless in branch 'dev/trunk'
        [ https://svn.apache.org/r1599292 ]

        LUCENE-5724: add CHANGES

        Show
        ASF subversion and git services added a comment - Commit 1599292 from Michael McCandless in branch 'dev/trunk' [ https://svn.apache.org/r1599292 ] LUCENE-5724 : add CHANGES
        Hide
        ASF subversion and git services added a comment -

        Commit 1599293 from Michael McCandless in branch 'dev/branches/branch_4x'
        [ https://svn.apache.org/r1599293 ]

        LUCENE-5724: add CHANGES

        Show
        ASF subversion and git services added a comment - Commit 1599293 from Michael McCandless in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1599293 ] LUCENE-5724 : add CHANGES

          People

          • Assignee:
            Michael McCandless
            Reporter:
            Michael McCandless
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development