Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-4084

unable to generate a 4.x back compat test index

        Details

        • Type: Bug
        • Status: Closed
        • Priority: Major
        • Resolution: Fixed
        • Affects Version/s: None
        • Fix Version/s: 4.0-ALPHA, 6.0
        • Component/s: core/index
        • Labels:
          None
        • Lucene Fields:
          New

          Description

          It seems the issue is an IWC cannot be used across 2 different indexwriters, because the DWPTThreadPool is created on the IndexWriterConfig, and you hit a SetOnce exception... seems like the exception is correct and it would be bad to have two indexwriters sharing the same DWPTThreadPool?

          stacktrace to follow in a comment.

          1. Text File
            LUCENE-4084.patch
            19 kB
            Michael McCandless
          2. Text File
            LUCENE-4084.patch
            20 kB
            Robert Muir
          3. Text File
            LUCENE-4084.patch
            21 kB
            Michael McCandless

            Activity

            Descending order - Click to sort in ascending order
            Hide
            Permalink
            rcmuir Robert Muir added a comment -

            looks good, +1

            Show
            rcmuir Robert Muir added a comment - looks good, +1
            Hide
            Permalink
            mikemccand Michael McCandless added a comment -

            New patch, starting from Rob's patch and adding changes.txt entry. I also cutover the IWC's SetOnce to AtomicBoolean, to record if this IWC is private to an IW to catch misuse.

            I think it's ready.

            Show
            mikemccand Michael McCandless added a comment - New patch, starting from Rob's patch and adding changes.txt entry. I also cutover the IWC's SetOnce to AtomicBoolean, to record if this IWC is private to an IW to catch misuse. I think it's ready.
            Hide
            Permalink
            mikemccand Michael McCandless added a comment -

            I think we need to add a changes entry too since MP now needs to impl clone in some cases?

            I agree, I'll add...

            Show
            mikemccand Michael McCandless added a comment - I think we need to add a changes entry too since MP now needs to impl clone in some cases? I agree, I'll add...
            Hide
            Permalink
            simonw Simon Willnauer added a comment -

            I think we need to add a changes entry too since MP now needs to impl clone in some cases?

            Show
            simonw Simon Willnauer added a comment - I think we need to add a changes entry too since MP now needs to impl clone in some cases?
            Hide
            Permalink
            mikemccand Michael McCandless added a comment -

            Basically the issue is that RandomDWPTThreadPool was public, but its pkg-private subclass had the new interface (Cloneable), and i think this confuses class-use etc etc.

            Aha! OK, thanks for fixing.... I think patch is ready now.

            Show
            mikemccand Michael McCandless added a comment - Basically the issue is that RandomDWPTThreadPool was public, but its pkg-private subclass had the new interface (Cloneable), and i think this confuses class-use etc etc. Aha! OK, thanks for fixing.... I think patch is ready now.
            Hide
            Permalink
            rcmuir Robert Muir added a comment -

            patch with javadocs-lint working.

            Basically the issue is that RandomDWPTThreadPool was public, but its pkg-private subclass had the new interface (Cloneable), and i think this confuses class-use etc etc.

            Show
            rcmuir Robert Muir added a comment - patch with javadocs-lint working. Basically the issue is that RandomDWPTThreadPool was public, but its pkg-private subclass had the new interface (Cloneable), and i think this confuses class-use etc etc.
            Hide
            Permalink
            mikemccand Michael McCandless added a comment -

            Patch.

            I fixed IWC.clone to do a deepish-clone for those members that have IW-specific state, and then fixed various tests that relied on those members not being cloned ....

            Tests pass, but the javadocs-lint is angry... not sure why yet.

            Show
            mikemccand Michael McCandless added a comment - Patch. I fixed IWC.clone to do a deepish-clone for those members that have IW-specific state, and then fixed various tests that relied on those members not being cloned .... Tests pass, but the javadocs-lint is angry... not sure why yet.
            Hide
            Permalink
            rcmuir Robert Muir added a comment -

            Yes:

            Additionally, I think its possible this might affect 3.x as well (I didnt test). depending on the fix maybe we can also backport if its appropriate.

            Today IWC documents that IW makes a private clone() so you can edit some settings live, but this IWC.clone() is shallow so this isnt really working.

            Show
            rcmuir Robert Muir added a comment - Yes: Additionally, I think its possible this might affect 3.x as well (I didnt test). depending on the fix maybe we can also backport if its appropriate. Today IWC documents that IW makes a private clone() so you can edit some settings live, but this IWC.clone() is shallow so this isnt really working.
            Hide
            Permalink
            simonw Simon Willnauer added a comment -

            same is true for MergePolicy no?

            Show
            simonw Simon Willnauer added a comment - same is true for MergePolicy no?
            Hide
            Permalink
            rcmuir Robert Muir added a comment - 
              [junit4] ERROR   0.23s | TestBackwardsCompatibility.testCreateCFS
   [junit4]    > Throwable #1: org.apache.lucene.util.SetOnce$AlreadySetException: The object cannot be set twice!
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([2ADDAAF27DE7C63D:38D013F5650FDD60]:0)
   [junit4]    > 	at org.apache.lucene.util.SetOnce.set(SetOnce.java:69)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriterPerThreadPool.initialize(DocumentsWriterPerThreadPool.java:139)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.<init>(DocumentsWriter.java:145)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:668)
   [junit4]    > 	at org.apache.lucene.index.TestBackwardsCompatibility.createIndex(TestBackwardsCompatibility.java:464)
   [junit4]    > 	at org.apache.lucene.index.TestBackwardsCompatibility.testCreateCFS(TestBackwardsCompatibility.java:78)
            Show
            rcmuir Robert Muir added a comment - [junit4] ERROR 0.23s | TestBackwardsCompatibility.testCreateCFS [junit4] > Throwable #1: org.apache.lucene.util.SetOnce$AlreadySetException: The object cannot be set twice! [junit4] > at __randomizedtesting.SeedInfo.seed([2ADDAAF27DE7C63D:38D013F5650FDD60]:0) [junit4] > at org.apache.lucene.util.SetOnce.set(SetOnce.java:69) [junit4] > at org.apache.lucene.index.DocumentsWriterPerThreadPool.initialize(DocumentsWriterPerThreadPool.java:139) [junit4] > at org.apache.lucene.index.DocumentsWriter.<init>(DocumentsWriter.java:145) [junit4] > at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:668) [junit4] > at org.apache.lucene.index.TestBackwardsCompatibility.createIndex(TestBackwardsCompatibility.java:464) [junit4] > at org.apache.lucene.index.TestBackwardsCompatibility.testCreateCFS(TestBackwardsCompatibility.java:78)

              People

              • Assignee:
                mikemccand Michael McCandless
                Reporter:
                rcmuir Robert Muir
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Development