Lucene - Core
  1. Lucene - Core
  2. LUCENE-6518

"classcache reaper" needs exemption from thread leaks

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.3, 6.0
    • Component/s: None
    • Labels:
    • Lucene Fields:
      New

      Description

      This happens on IBM JDK (some system thread or what have you).

      its 100% reproducible if you just cd lucene/analysis/smartcn and run 'ant test': we time out waiting for the thread to die and fail like this:

         [junit4] Suite: org.apache.lucene.analysis.cn.smart.TestHMMChineseTokenizerFactory
         [junit4]   2> Jun 03, 2015 12:30:39 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
         [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
         [junit4]   2> Jun 03, 2015 12:30:59 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
         [junit4]   2> SEVERE: 1 thread leaked from SUITE scope at org.apache.lucene.analysis.cn.smart.TestHMMChineseTokenizerFactory: 
         [junit4]   2>    1) Thread[id=27, name=ClassCache Reaper, state=WAITING, group=TGRP-TestHMMChineseTokenizerFactory]
         [junit4]   2>         at java.lang.Object.wait(Native Method)
         [junit4]   2>         at java.lang.Object.wait(Object.java:201)
         [junit4]   2>         at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:119)
         [junit4]   2>         at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:91)
         [junit4]   2>         at java.io.ClassCache$Reaper.run(ClassCache.java:222)
         [junit4]   2> Jun 03, 2015 12:30:59 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
         [junit4]   2> INFO: Starting to interrupt leaked threads:
         [junit4]   2>    1) Thread[id=27, name=ClassCache Reaper, state=WAITING, group=TGRP-TestHMMChineseTokenizerFactory]
         [junit4]   2> Jun 03, 2015 12:31:02 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
         [junit4]   2> SEVERE: There are still zombie threads that couldn't be terminated:
         [junit4]   2>    1) Thread[id=27, name=ClassCache Reaper, state=WAITING, group=TGRP-TestHMMChineseTokenizerFactory]
         [junit4]   2>         at java.lang.Object.wait(Native Method)
         [junit4]   2>         at java.lang.Object.wait(Object.java:201)
         [junit4]   2>         at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:119)
         [junit4]   2>         at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:91)
         [junit4]   2>         at java.io.ClassCache$Reaper.run(ClassCache.java:222)
         [junit4]   2> NOTE: test params are: codec=Asserting(Lucene50): {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=ar_JO, timezone=Europe/Andorra
         [junit4]   2> NOTE: Linux 3.13.0-49-generic amd64/IBM Corporation 1.8.0 (64-bit)/cpus=8,threads=2,free=1204464,total=20643840
         [junit4]   2> NOTE: All tests run in this JVM: [TestHMMChineseTokenizerFactory]
         [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestHMMChineseTokenizerFactory -Dtests.seed=4546D15E2A47A285 -Dtests.locale=ar_JO -Dtests.timezone=Europe/Andorra -Dtests.asserts=true -Dtests.file.encoding=UTF-8
      
      1. LUCENE-6518.patch
        1.0 kB
        Robert Muir
      2. LUCENE-6518.patch
        0.8 kB
        Dawid Weiss

        Activity

        Hide
        Robert Muir added a comment -

        fyi Dawid Weiss as I don't know if this is something we should fix in randomizedtesting or in lucene test infra. I know there is a "list" somewhere for stuff like this.

        Show
        Robert Muir added a comment - fyi Dawid Weiss as I don't know if this is something we should fix in randomizedtesting or in lucene test infra. I know there is a "list" somewhere for stuff like this.
        Hide
        Dawid Weiss added a comment -

        This class is used to exclude system/ known/ odd threads.

        Show
        Dawid Weiss added a comment - This class is used to exclude system/ known/ odd threads.
        Hide
        Dawid Weiss added a comment -

        This class's source is even provided in IBM's jdk. Interesting. The QuickPatchThreadsFilter has a list of ignored threads. This is by large hard to predict, so I'd just include it there.

        Show
        Dawid Weiss added a comment - This class's source is even provided in IBM's jdk. Interesting. The QuickPatchThreadsFilter has a list of ignored threads. This is by large hard to predict, so I'd just include it there.
        Hide
        Robert Muir added a comment -

        +1 Dawid, thank you for looking at this.

        Show
        Robert Muir added a comment - +1 Dawid, thank you for looking at this.
        Hide
        Robert Muir added a comment -

        Dawid, I had to tweak your J9 detection, this is the logic i've used in tests before.

        I will commit this and iterate on the other silly issues, thanks for the patch!

        Show
        Robert Muir added a comment - Dawid, I had to tweak your J9 detection, this is the logic i've used in tests before. I will commit this and iterate on the other silly issues, thanks for the patch!
        Hide
        ASF subversion and git services added a comment -

        Commit 1683451 from Robert Muir in branch 'dev/trunk'
        [ https://svn.apache.org/r1683451 ]

        LUCENE-6518: classcache reaper needs exemption from thread leaks (J9)

        Show
        ASF subversion and git services added a comment - Commit 1683451 from Robert Muir in branch 'dev/trunk' [ https://svn.apache.org/r1683451 ] LUCENE-6518 : classcache reaper needs exemption from thread leaks (J9)
        Hide
        ASF subversion and git services added a comment -

        Commit 1683452 from Robert Muir in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1683452 ]

        LUCENE-6518: classcache reaper needs exemption from thread leaks (J9)

        Show
        ASF subversion and git services added a comment - Commit 1683452 from Robert Muir in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1683452 ] LUCENE-6518 : classcache reaper needs exemption from thread leaks (J9)
        Hide
        Shalin Shekhar Mangar added a comment -

        Bulk close for 5.3.0 release

        Show
        Shalin Shekhar Mangar added a comment - Bulk close for 5.3.0 release

          People

          • Assignee:
            Unassigned
            Reporter:
            Robert Muir
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development