Lucene - Core
  1. Lucene - Core
  2. LUCENE-6478

Test execution can hang with java.security.debug

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.4, 6.0
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      As reported by Robert:

      # clone trunk
      cd lucene/core/
      ant test -Dargs="-Djava.security.debug=access:failure" -Dtestcase=TestDemo
      

      Hangs the test runner. The same problem appears to be present in ES builds too. It seems like some kind of weird stream buffer problem, the security framework seems to be writing to the native descriptors directly. Will have to dig (deep...).

        Issue Links

          Activity

          Hide
          Dawid Weiss added a comment -

          I know what's causing this problem. I don't know if I can fix it, but I'll try.

          Show
          Dawid Weiss added a comment - I know what's causing this problem. I don't know if I can fix it, but I'll try.
          Hide
          Dawid Weiss added a comment -

          Note how recursive interaction between the security framework and gson is the problem here:

          WARN: Unhandled exception in event serialization. -> java.lang.IllegalStateException
                  at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.Gson$FutureTypeAdapter.write(Gson.java:897)
                  at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.Gson.toJson(Gson.java:600)
                  at com.carrotsearch.ant.tasks.junit4.events.Serializer.flushQueue(Serializer.java:100)
                  at com.carrotsearch.ant.tasks.junit4.events.Serializer.serialize(Serializer.java:81)
                  at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain$4.write(SlaveMain.java:434)
                  at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
                  at java.io.BufferedOutputStream.write(BufferedOutputStream.java:126)
                  at java.io.PrintStream.write(PrintStream.java:480)
                  at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
                  at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
                  at sun.nio.cs.StreamEncoder.flushBuffer(StreamEncoder.java:104)
                  at java.io.OutputStreamWriter.flushBuffer(OutputStreamWriter.java:185)
                  at java.io.PrintStream.write(PrintStream.java:527)
                  at java.io.PrintStream.print(PrintStream.java:669)
                  at java.io.PrintStream.println(PrintStream.java:806)
                  at sun.security.util.Debug.println(Debug.java:162)
                  at java.security.AccessControlContext.checkPermission(AccessControlContext.java:463)
                  at java.security.AccessController.checkPermission(AccessController.java:884)
                  at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
                  at java.lang.reflect.AccessibleObject.setAccessible(AccessibleObject.java:128)
                  at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.internal.ConstructorConstructor.newDefaultCons
          tructor(ConstructorConstructor.java:97)
                  at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.internal.ConstructorConstructor.get(Constructo
          rConstructor.java:79)
                  at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.cre
          ate(ReflectiveTypeAdapterFactory.java:82)
                  at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.Gson.getAdapter(Gson.java:359)
                  at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.Gson.toJson(Gson.java:592)
                  at com.carrotsearch.ant.tasks.junit4.events.Serializer.flushQueue(Serializer.java:100)
                  at com.carrotsearch.ant.tasks.junit4.events.Serializer.serialize(Serializer.java:81)
                  at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain$4.write(SlaveMain.java:434)
                  at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
                  at java.io.BufferedOutputStream.write(BufferedOutputStream.java:126)
                  at java.io.PrintStream.write(PrintStream.java:480)
                  at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
                  at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
                  at sun.nio.cs.StreamEncoder.flushBuffer(StreamEncoder.java:104)
                  at java.io.OutputStreamWriter.flushBuffer(OutputStreamWriter.java:185)
                  at java.io.PrintStream.write(PrintStream.java:527)
                  at java.io.PrintStream.print(PrintStream.java:669)
                  at java.io.PrintStream.println(PrintStream.java:806)
                  at sun.security.util.Debug.println(Debug.java:162)
                  at java.security.AccessController.checkPermission(AccessController.java:878)
                  at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
                  at java.lang.SecurityManager.checkRead(SecurityManager.java:888)
                  at java.io.File.exists(File.java:814)
                  at java.io.WinNTFileSystem.canonicalize(WinNTFileSystem.java:434)
                  at java.io.File.getCanonicalPath(File.java:618)
                  at java.io.FilePermission$1.run(FilePermission.java:215)
                  at java.io.FilePermission$1.run(FilePermission.java:203)
                  at java.security.AccessController.doPrivileged(Native Method)
                  at java.io.FilePermission.init(FilePermission.java:203)
                  at java.io.FilePermission.<init>(FilePermission.java:277)
                  at java.lang.SecurityManager.checkRead(SecurityManager.java:888)
                  at java.io.File.exists(File.java:814)
                  at sun.misc.URLClassPath$FileLoader.getResource(URLClassPath.java:1081)
                  at sun.misc.URLClassPath.getResource(URLClassPath.java:199)
                  at java.net.URLClassLoader$1.run(URLClassLoader.java:364)
                  at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
                  at java.security.AccessController.doPrivileged(Native Method)
                  at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
                  at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
                  at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
                  at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
                  at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:330)
                  at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:14)
          

          Just about any option I can think of is using plain manual serialization (sigh) of events and/or disabling sysout capture.

          Show
          Dawid Weiss added a comment - Note how recursive interaction between the security framework and gson is the problem here: WARN: Unhandled exception in event serialization. -> java.lang.IllegalStateException at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.Gson$FutureTypeAdapter.write(Gson.java:897) at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.Gson.toJson(Gson.java:600) at com.carrotsearch.ant.tasks.junit4.events.Serializer.flushQueue(Serializer.java:100) at com.carrotsearch.ant.tasks.junit4.events.Serializer.serialize(Serializer.java:81) at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain$4.write(SlaveMain.java:434) at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) at java.io.BufferedOutputStream.write(BufferedOutputStream.java:126) at java.io.PrintStream.write(PrintStream.java:480) at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221) at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291) at sun.nio.cs.StreamEncoder.flushBuffer(StreamEncoder.java:104) at java.io.OutputStreamWriter.flushBuffer(OutputStreamWriter.java:185) at java.io.PrintStream.write(PrintStream.java:527) at java.io.PrintStream.print(PrintStream.java:669) at java.io.PrintStream.println(PrintStream.java:806) at sun.security.util.Debug.println(Debug.java:162) at java.security.AccessControlContext.checkPermission(AccessControlContext.java:463) at java.security.AccessController.checkPermission(AccessController.java:884) at java.lang. SecurityManager .checkPermission( SecurityManager .java:549) at java.lang.reflect.AccessibleObject.setAccessible(AccessibleObject.java:128) at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.internal.ConstructorConstructor.newDefaultCons tructor(ConstructorConstructor.java:97) at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.internal.ConstructorConstructor.get(Constructo rConstructor.java:79) at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.cre ate(ReflectiveTypeAdapterFactory.java:82) at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.Gson.getAdapter(Gson.java:359) at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.Gson.toJson(Gson.java:592) at com.carrotsearch.ant.tasks.junit4.events.Serializer.flushQueue(Serializer.java:100) at com.carrotsearch.ant.tasks.junit4.events.Serializer.serialize(Serializer.java:81) at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain$4.write(SlaveMain.java:434) at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) at java.io.BufferedOutputStream.write(BufferedOutputStream.java:126) at java.io.PrintStream.write(PrintStream.java:480) at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221) at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291) at sun.nio.cs.StreamEncoder.flushBuffer(StreamEncoder.java:104) at java.io.OutputStreamWriter.flushBuffer(OutputStreamWriter.java:185) at java.io.PrintStream.write(PrintStream.java:527) at java.io.PrintStream.print(PrintStream.java:669) at java.io.PrintStream.println(PrintStream.java:806) at sun.security.util.Debug.println(Debug.java:162) at java.security.AccessController.checkPermission(AccessController.java:878) at java.lang. SecurityManager .checkPermission( SecurityManager .java:549) at java.lang. SecurityManager .checkRead( SecurityManager .java:888) at java.io.File.exists(File.java:814) at java.io.WinNTFileSystem.canonicalize(WinNTFileSystem.java:434) at java.io.File.getCanonicalPath(File.java:618) at java.io.FilePermission$1.run(FilePermission.java:215) at java.io.FilePermission$1.run(FilePermission.java:203) at java.security.AccessController.doPrivileged(Native Method) at java.io.FilePermission.init(FilePermission.java:203) at java.io.FilePermission.<init>(FilePermission.java:277) at java.lang. SecurityManager .checkRead( SecurityManager .java:888) at java.io.File.exists(File.java:814) at sun.misc.URLClassPath$FileLoader.getResource(URLClassPath.java:1081) at sun.misc.URLClassPath.getResource(URLClassPath.java:199) at java.net.URLClassLoader$1.run(URLClassLoader.java:364) at java.net.URLClassLoader$1.run(URLClassLoader.java:361) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:360) at java.lang. ClassLoader .loadClass( ClassLoader .java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang. ClassLoader .loadClass( ClassLoader .java:357) at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:330) at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:14) Just about any option I can think of is using plain manual serialization (sigh) of events and/or disabling sysout capture.
          Hide
          Dawid Weiss added a comment - - edited

          Good news, Robert (Robert Muir). I think I fixed it in the RR – I was successfull at running tests with full security policy dump.

          cd lucene
          ant test-core -Dtests.useSecurityManager=true  "-Dtests.class=*FuzzyTermOnShortTermsTest*" -Dtests.seed=deadbeef -Dtests.jvms=1 -Dtests.verbose=true -Dargs="-Djava.security.debug=all"
          

          You need -Dtests.verbose – there's tons of output from policy debug and it max sysout limit rule complains otherwise. I attach a patch that I used against trunk. Note it uses a snapshot version of RR, you need to fetch it from sonatype snapshots repo (or install it manually in your .m2).

          Index: ivy-settings.xml
          ===================================================================
          --- ivy-settings.xml    (revision 1710751)
          +++ ivy-settings.xml    (working copy)
          @@ -50,7 +50,7 @@
          
               <chain name="default" returnFirst="true" checkmodified="true" changingPattern=".*SNAPSHOT">
                 <resolver ref="local"/>
          -      <!-- <resolver ref="local-maven-2" /> -->
          +      <resolver ref="local-maven-2" />
                 <resolver ref="main"/>
                 <resolver ref="maven.restlet.org" />
                 <resolver ref="sonatype-releases" />
          Index: ivy-versions.properties
          ===================================================================
          --- ivy-versions.properties     (revision 1710751)
          +++ ivy-versions.properties     (working copy)
          @@ -7,7 +7,7 @@
           /cglib/cglib-nodep = 2.2
           /com.adobe.xmp/xmpcore = 5.1.2
          
          -com.carrotsearch.randomizedtesting.version = 2.1.17
          +com.carrotsearch.randomizedtesting.version = 2.2.0-SNAPSHOT
           /com.carrotsearch.randomizedtesting/junit4-ant = ${com.carrotsearch.randomizedtesting.version}
           /com.carrotsearch.randomizedtesting/randomizedtesting-runner = ${com.carrotsearch.randomizedtesting.version}
          
          Show
          Dawid Weiss added a comment - - edited Good news, Robert ( Robert Muir ). I think I fixed it in the RR – I was successfull at running tests with full security policy dump. cd lucene ant test-core -Dtests.useSecurityManager= true "-Dtests.class=*FuzzyTermOnShortTermsTest*" -Dtests.seed=deadbeef -Dtests.jvms=1 -Dtests.verbose= true -Dargs= "-Djava.security.debug=all" You need -Dtests.verbose – there's tons of output from policy debug and it max sysout limit rule complains otherwise. I attach a patch that I used against trunk. Note it uses a snapshot version of RR , you need to fetch it from sonatype snapshots repo (or install it manually in your .m2 ). Index: ivy-settings.xml =================================================================== --- ivy-settings.xml (revision 1710751) +++ ivy-settings.xml (working copy) @@ -50,7 +50,7 @@ <chain name= " default " returnFirst= " true " checkmodified= " true " changingPattern= ".*SNAPSHOT" > <resolver ref= "local" /> - <!-- <resolver ref= "local-maven-2" /> --> + <resolver ref= "local-maven-2" /> <resolver ref= "main" /> <resolver ref= "maven.restlet.org" /> <resolver ref= "sonatype-releases" /> Index: ivy-versions.properties =================================================================== --- ivy-versions.properties (revision 1710751) +++ ivy-versions.properties (working copy) @@ -7,7 +7,7 @@ /cglib/cglib-nodep = 2.2 /com.adobe.xmp/xmpcore = 5.1.2 -com.carrotsearch.randomizedtesting.version = 2.1.17 +com.carrotsearch.randomizedtesting.version = 2.2.0-SNAPSHOT /com.carrotsearch.randomizedtesting/junit4-ant = ${com.carrotsearch.randomizedtesting.version} /com.carrotsearch.randomizedtesting/randomizedtesting-runner = ${com.carrotsearch.randomizedtesting.version}
          Hide
          Robert Muir added a comment -

          this is really great: I tested it and it works.

          without this fix, if you have an issue then debugging has to be....creatively done.

          Show
          Robert Muir added a comment - this is really great: I tested it and it works. without this fix, if you have an issue then debugging has to be....creatively done.
          Hide
          Dawid Weiss added a comment -

          There will be a few more changes to RR 2.2.0, but I plan to release soon-ish. Hopefully nothing will break.

          Show
          Dawid Weiss added a comment - There will be a few more changes to RR 2.2.0, but I plan to release soon-ish. Hopefully nothing will break.

            People

            • Assignee:
              Dawid Weiss
              Reporter:
              Dawid Weiss
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development