Lucene - Core
  1. Lucene - Core
  2. LUCENE-4084

unable to generate a 4.x back compat test index

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.0-ALPHA, 6.0
    • Component/s: core/index
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      It seems the issue is an IWC cannot be used across 2 different indexwriters, because the DWPTThreadPool is created on the IndexWriterConfig, and you hit a SetOnce exception... seems like the exception is correct and it would be bad to have two indexwriters sharing the same DWPTThreadPool?

      stacktrace to follow in a comment.

      1. LUCENE-4084.patch
        21 kB
        Michael McCandless
      2. LUCENE-4084.patch
        20 kB
        Robert Muir
      3. LUCENE-4084.patch
        19 kB
        Michael McCandless

        Activity

        Hide
        Robert Muir added a comment -
          [junit4] ERROR   0.23s | TestBackwardsCompatibility.testCreateCFS
           [junit4]    > Throwable #1: org.apache.lucene.util.SetOnce$AlreadySetException: The object cannot be set twice!
           [junit4]    > 	at __randomizedtesting.SeedInfo.seed([2ADDAAF27DE7C63D:38D013F5650FDD60]:0)
           [junit4]    > 	at org.apache.lucene.util.SetOnce.set(SetOnce.java:69)
           [junit4]    > 	at org.apache.lucene.index.DocumentsWriterPerThreadPool.initialize(DocumentsWriterPerThreadPool.java:139)
           [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.<init>(DocumentsWriter.java:145)
           [junit4]    > 	at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:668)
           [junit4]    > 	at org.apache.lucene.index.TestBackwardsCompatibility.createIndex(TestBackwardsCompatibility.java:464)
           [junit4]    > 	at org.apache.lucene.index.TestBackwardsCompatibility.testCreateCFS(TestBackwardsCompatibility.java:78)
        
        Show
        Robert Muir added a comment - [junit4] ERROR 0.23s | TestBackwardsCompatibility.testCreateCFS [junit4] > Throwable #1: org.apache.lucene.util.SetOnce$AlreadySetException: The object cannot be set twice! [junit4] > at __randomizedtesting.SeedInfo.seed([2ADDAAF27DE7C63D:38D013F5650FDD60]:0) [junit4] > at org.apache.lucene.util.SetOnce.set(SetOnce.java:69) [junit4] > at org.apache.lucene.index.DocumentsWriterPerThreadPool.initialize(DocumentsWriterPerThreadPool.java:139) [junit4] > at org.apache.lucene.index.DocumentsWriter.<init>(DocumentsWriter.java:145) [junit4] > at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:668) [junit4] > at org.apache.lucene.index.TestBackwardsCompatibility.createIndex(TestBackwardsCompatibility.java:464) [junit4] > at org.apache.lucene.index.TestBackwardsCompatibility.testCreateCFS(TestBackwardsCompatibility.java:78)
        Hide
        Simon Willnauer added a comment -

        same is true for MergePolicy no?

        Show
        Simon Willnauer added a comment - same is true for MergePolicy no?
        Hide
        Robert Muir added a comment -

        Yes:

        Additionally, I think its possible this might affect 3.x as well (I didnt test). depending on the fix maybe we can also backport if its appropriate.

        Today IWC documents that IW makes a private clone() so you can edit some settings live, but this IWC.clone() is shallow so this isnt really working.

        Show
        Robert Muir added a comment - Yes: Additionally, I think its possible this might affect 3.x as well (I didnt test). depending on the fix maybe we can also backport if its appropriate. Today IWC documents that IW makes a private clone() so you can edit some settings live, but this IWC.clone() is shallow so this isnt really working.
        Hide
        Michael McCandless added a comment -

        Patch.

        I fixed IWC.clone to do a deepish-clone for those members that have IW-specific state, and then fixed various tests that relied on those members not being cloned ....

        Tests pass, but the javadocs-lint is angry... not sure why yet.

        Show
        Michael McCandless added a comment - Patch. I fixed IWC.clone to do a deepish-clone for those members that have IW-specific state, and then fixed various tests that relied on those members not being cloned .... Tests pass, but the javadocs-lint is angry... not sure why yet.
        Hide
        Robert Muir added a comment -

        patch with javadocs-lint working.

        Basically the issue is that RandomDWPTThreadPool was public, but its pkg-private subclass had the new interface (Cloneable), and i think this confuses class-use etc etc.

        Show
        Robert Muir added a comment - patch with javadocs-lint working. Basically the issue is that RandomDWPTThreadPool was public, but its pkg-private subclass had the new interface (Cloneable), and i think this confuses class-use etc etc.
        Hide
        Michael McCandless added a comment -

        Basically the issue is that RandomDWPTThreadPool was public, but its pkg-private subclass had the new interface (Cloneable), and i think this confuses class-use etc etc.

        Aha! OK, thanks for fixing.... I think patch is ready now.

        Show
        Michael McCandless added a comment - Basically the issue is that RandomDWPTThreadPool was public, but its pkg-private subclass had the new interface (Cloneable), and i think this confuses class-use etc etc. Aha! OK, thanks for fixing.... I think patch is ready now.
        Hide
        Simon Willnauer added a comment -

        I think we need to add a changes entry too since MP now needs to impl clone in some cases?

        Show
        Simon Willnauer added a comment - I think we need to add a changes entry too since MP now needs to impl clone in some cases?
        Hide
        Michael McCandless added a comment -

        I think we need to add a changes entry too since MP now needs to impl clone in some cases?

        I agree, I'll add...

        Show
        Michael McCandless added a comment - I think we need to add a changes entry too since MP now needs to impl clone in some cases? I agree, I'll add...
        Hide
        Michael McCandless added a comment -

        New patch, starting from Rob's patch and adding changes.txt entry. I also cutover the IWC's SetOnce to AtomicBoolean, to record if this IWC is private to an IW to catch misuse.

        I think it's ready.

        Show
        Michael McCandless added a comment - New patch, starting from Rob's patch and adding changes.txt entry. I also cutover the IWC's SetOnce to AtomicBoolean, to record if this IWC is private to an IW to catch misuse. I think it's ready.
        Hide
        Robert Muir added a comment -

        looks good, +1

        Show
        Robert Muir added a comment - looks good, +1

          People

          • Assignee:
            Michael McCandless
            Reporter:
            Robert Muir
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development