Lucene - Core
  1. Lucene - Core
  2. LUCENE-5086

RamUsageEstimator causes AWT classes to be loaded by calling ManagementFactory#getPlatformMBeanServer

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.4, 6.0
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      Yea, that type of day and that type of title .

      Since the last update of Java 6 on OS X, I started to see an annoying icon pop up at the doc whenever running elasticsearch. By default, all of our scripts add headless AWT flag so people will probably not encounter it, but, it was strange that I saw it when before I didn't.

      I started to dig around, and saw that when RamUsageEstimator was being loaded, it was causing AWT classes to be loaded. Further investigation showed that actually for some reason, calling ManagementFactory#getPlatformMBeanServer now with the new Java version causes AWT classes to be loaded (at least on the mac, haven't tested on other platforms yet).

      There are several ways to try and solve it, for example, by identifying the bug in the JVM itself, but I think that there should be a fix for it in Lucene itself, specifically since there is no need to call #getPlatformMBeanServer to get the hotspot diagnostics one (its a heavy call...).

      Here is a simple call that will allow to get the hotspot mxbean without using the #getPlatformMBeanServer method, and not causing it to be loaded and loading all those nasty AWT classes:

          Object getHotSpotMXBean() {
              try {
                  // Java 6
                  Class sunMF = Class.forName("sun.management.ManagementFactory");
                  return sunMF.getMethod("getDiagnosticMXBean").invoke(null);
              } catch (Throwable t) {
                  // ignore
              }
              // potentially Java 7
              try {
                  return ManagementFactory.class.getMethod("getPlatformMXBean", Class.class).invoke(null, Class.forName("com.sun.management.HotSpotDiagnosticMXBean"));
              } catch (Throwable t) {
                  // ignore
              }
              return null;
          }
      
      1. LUCENE-5086.patch
        2 kB
        Dawid Weiss
      2. LUCENE-5086-branch4x.patch
        2 kB
        Uwe Schindler
      3. LUCENE-5086-branch4x.patch
        2 kB
        Uwe Schindler
      4. LUCENE-5086-trunk.patch
        2 kB
        Uwe Schindler
      5. LUCENE-5086-trunk.patch
        1 kB
        Uwe Schindler

        Issue Links

          Activity

          Hide
          Robert Muir added a comment -

          LOL

          Show
          Robert Muir added a comment - LOL
          Hide
          Dawid Weiss added a comment -

          This would be normally handled by the test framework as a thread leak, but it's explicitly excluded in default randomizedtesting filters:

                // Explicit check for MacOSX AWT-AppKit
                if (t.getName().equals("AWT-AppKit")) {
                  return true;
                }
          

          I believe ManagementFactory#getPlatformMBeanServer was not the only call that could have started that AWT daemon, very odd.

          Show
          Dawid Weiss added a comment - This would be normally handled by the test framework as a thread leak, but it's explicitly excluded in default randomizedtesting filters: // Explicit check for MacOSX AWT-AppKit if (t.getName().equals( "AWT-AppKit" )) { return true ; } I believe ManagementFactory#getPlatformMBeanServer was not the only call that could have started that AWT daemon, very odd.
          Hide
          Dawid Weiss added a comment -

          Shay, can you say which version of Java (macosx) is causing this to happen? I'll try to reproduce tonight on my mac and see what the possible workarounds are (yours included).

          Show
          Dawid Weiss added a comment - Shay, can you say which version of Java (macosx) is causing this to happen? I'll try to reproduce tonight on my mac and see what the possible workarounds are (yours included).
          Hide
          Uwe Schindler added a comment - - edited

          I updated MacOSX to 1.6.0_51 a few days ago on the Jenkins Server, but no popups

          Apple's MacOSX Java also starts the AWT thread when loading the Java scripting framework, because there is one scripting language in the SPI list (something like ÄppleFooBarScriptingEngine which also initializes AWT, also ignoring awt.headless=true)... It's all horrible.

          Show
          Uwe Schindler added a comment - - edited I updated MacOSX to 1.6.0_51 a few days ago on the Jenkins Server, but no popups Apple's MacOSX Java also starts the AWT thread when loading the Java scripting framework, because there is one scripting language in the SPI list (something like ÄppleFooBarScriptingEngine which also initializes AWT, also ignoring awt.headless=true)... It's all horrible.
          Hide
          Uwe Schindler added a comment -

          We can use the above code fragment to load the MXBean, if it can fall back to the official code when the direct instantiation does not work. And it should never ever catch Throwable, please specify the exact Exceptions and handle accordingly. On Trunk you can use the multi-catch as Lucene trunk is Java 7.

          Show
          Uwe Schindler added a comment - We can use the above code fragment to load the MXBean, if it can fall back to the official code when the direct instantiation does not work. And it should never ever catch Throwable, please specify the exact Exceptions and handle accordingly. On Trunk you can use the multi-catch as Lucene trunk is Java 7.
          Hide
          Shay Banon added a comment -

          The Java version on the Mac is the latest one:

          java version "1.6.0_51"
          Java(TM) SE Runtime Environment (build 1.6.0_51-b11-457-11M4509)
          Java HotSpot(TM) 64-Bit Server VM (build 20.51-b01-457, mixed mode)

          Regarding the catch, I think Throwable is the right exceptions to catch here. Catch all, who cares, you don't want a bug in the JVM that throws an unexpected runtime exception to cause Lucene to break the APP completely because its a static block, and I have been right there a few times. But if you feel differently, go ahead and change it to explicitly catch whats needed.

          Show
          Shay Banon added a comment - The Java version on the Mac is the latest one: java version "1.6.0_51" Java(TM) SE Runtime Environment (build 1.6.0_51-b11-457-11M4509) Java HotSpot(TM) 64-Bit Server VM (build 20.51-b01-457, mixed mode) Regarding the catch, I think Throwable is the right exceptions to catch here. Catch all, who cares, you don't want a bug in the JVM that throws an unexpected runtime exception to cause Lucene to break the APP completely because its a static block, and I have been right there a few times. But if you feel differently, go ahead and change it to explicitly catch whats needed.
          Hide
          Uwe Schindler added a comment -

          I agree, the other code in RAMUsageEstimator catches Exception, but not Throwable. I would use a similar chain of calls in RamUsageEstimator.

          The Lucene tests use headless=true by default so are not affected by this.

          I will provide a patch later.

          Show
          Uwe Schindler added a comment - I agree, the other code in RAMUsageEstimator catches Exception, but not Throwable. I would use a similar chain of calls in RamUsageEstimator. The Lucene tests use headless=true by default so are not affected by this. I will provide a patch later.
          Hide
          Dawid Weiss added a comment -

          I remember about it but our mac died yesterday and my wife left the charger at work. I'll look into it tonight, unless Uwe is faster.

          Show
          Dawid Weiss added a comment - I remember about it but our mac died yesterday and my wife left the charger at work. I'll look into it tonight, unless Uwe is faster.
          Hide
          Dawid Weiss added a comment -

          Yeah, I confirm the problem Instead of applying the patch mentioned by Shay I'd like to spend some time and see what Alex Shipilev did in his tool.

          He's guessing the size of the alignment here:

          https://github.com/shipilev/java-object-layout/blob/master/src/main/java/org/openjdk/tools/objectlayout/VMSupport.java#L240

          but also reading it directly here (hotspot and jrockit separately):

          https://github.com/shipilev/java-object-layout/blob/master/src/main/java/org/openjdk/tools/objectlayout/VMSupport.java#L138

          I'll dig what he actually does and whether the results are in sync with what we have. He still calls getPlatformMBeanServer directly so there'll have to be a workaround for that anyway.

          Show
          Dawid Weiss added a comment - Yeah, I confirm the problem Instead of applying the patch mentioned by Shay I'd like to spend some time and see what Alex Shipilev did in his tool. He's guessing the size of the alignment here: https://github.com/shipilev/java-object-layout/blob/master/src/main/java/org/openjdk/tools/objectlayout/VMSupport.java#L240 but also reading it directly here (hotspot and jrockit separately): https://github.com/shipilev/java-object-layout/blob/master/src/main/java/org/openjdk/tools/objectlayout/VMSupport.java#L138 I'll dig what he actually does and whether the results are in sync with what we have. He still calls getPlatformMBeanServer directly so there'll have to be a workaround for that anyway.
          Hide
          Dawid Weiss added a comment -

          Just so that it doesn't escape: this patch does not bring up the AWT icon in the dock for me. I checked on a Mac with the default java (1.6).

          I will still look into Alex's estimation of the alignment property but if you want to commit this in then I can file another issue.

          Show
          Dawid Weiss added a comment - Just so that it doesn't escape: this patch does not bring up the AWT icon in the dock for me. I checked on a Mac with the default java (1.6). I will still look into Alex's estimation of the alignment property but if you want to commit this in then I can file another issue.
          Hide
          Uwe Schindler added a comment -

          +1

          Looks like the code I started this afternoon. In Lucene trunk (Java 7) we can remove the first check. We should check the code with all kniwn jvms and check which part is the source of yhe info (maybe using some printlns).

          Show
          Uwe Schindler added a comment - +1 Looks like the code I started this afternoon. In Lucene trunk (Java 7) we can remove the first check. We should check the code with all kniwn jvms and check which part is the source of yhe info (maybe using some printlns).
          Hide
          Dawid Weiss added a comment -

          Yep, I'll add a table to the issue. Going to bed now but will test tomorrow.

          Show
          Dawid Weiss added a comment - Yep, I'll add a table to the issue. Going to bed now but will test tomorrow.
          Hide
          Uwe Schindler added a comment -

          You can move: final Class<?> beanClazz = Class.forName("com.sun.management.HotSpotDiagnosticMXBean"); before the second try-block because when this fails its also too late for the "official way".

          Show
          Uwe Schindler added a comment - You can move: final Class<?> beanClazz = Class.forName("com.sun.management.HotSpotDiagnosticMXBean"); before the second try-block because when this fails its also too late for the "official way".
          Hide
          Uwe Schindler added a comment -

          Simplified patch for 4.x

          Show
          Uwe Schindler added a comment - Simplified patch for 4.x
          Hide
          Uwe Schindler added a comment -

          On Lucene trunk (Java 7) we can remove all try-catch blocks and only use to return the bean:

          final Class<?> beanClazz = Class.forName("com.sun.management.HotSpotDiagnosticMXBean");
          return ManagementFactory.getPlatformMXBean(beanClazz);
          

          The only one that needs to be reflected is the beanClazz, if it fails we are not on Hotspot (or something that does not emulate it).

          Show
          Uwe Schindler added a comment - On Lucene trunk (Java 7) we can remove all try-catch blocks and only use to return the bean: final Class <?> beanClazz = Class .forName( "com.sun.management.HotSpotDiagnosticMXBean" ); return ManagementFactory.getPlatformMXBean(beanClazz); The only one that needs to be reflected is the beanClazz, if it fails we are not on Hotspot (or something that does not emulate it).
          Hide
          Uwe Schindler added a comment -

          This is the patch for Java 7+ (using public API in ManagementFactory to get platform bean)

          Show
          Uwe Schindler added a comment - This is the patch for Java 7+ (using public API in ManagementFactory to get platform bean)
          Hide
          Uwe Schindler added a comment -

          Patch for branch4x (Lucene 4.x)

          Show
          Uwe Schindler added a comment - Patch for branch4x (Lucene 4.x)
          Hide
          Uwe Schindler added a comment -

          The given patches for trunk and 4.x seem to work correctly. I changed the 4.x patch a little bit and removed extra Class.forName's and reordered the catch clauses. It now first tries to use the "official" java 7 API and only if not on Java 7 falls back to the Java 6 thing. Maybe we can use Constants to detect the JDK earlier without trying.

          From my perspective the 2 patches fix the issue perfectly, the trunk code is the official Java7 way to get this bean without a RPC server/proxy overhead and now AWT

          Show
          Uwe Schindler added a comment - The given patches for trunk and 4.x seem to work correctly. I changed the 4.x patch a little bit and removed extra Class.forName's and reordered the catch clauses. It now first tries to use the "official" java 7 API and only if not on Java 7 falls back to the Java 6 thing. Maybe we can use Constants to detect the JDK earlier without trying. From my perspective the 2 patches fix the issue perfectly, the trunk code is the official Java7 way to get this bean without a RPC server/proxy overhead and now AWT
          Hide
          Uwe Schindler added a comment - - edited

          Some restructuring and preventing a useless NPE that needs to be catched on non-Hotspot JVMs.
          I also removed the separate private method and made the whole thing with nested catches like at the other places in the static {} initializer (I don't like class' methods called from a static initializer). Also this one is easier to debug, as you can add printlns without working around the "return" escape of the original method!

          I tested on Windows:

          • Oracle JDK 6u32 and 6u45 (64 bit) work, proprietary sunMF API used
          • Oracle JDK 6u32, 32 bit is unsupported as before (the object alignment JVM option does not exist at all, but also the sunMF API is used)
          • Oracle JDK 7u25 and Oracle JDK 8 preview (64 bit) works, new public Java 7 API is used (via reflection) or statically typed in trunk
          • All other JVMs are unsupported, but they don't have obejct alignment available

          By the way: Our code looks better and more universal, because it does not rely on the management bean to detect compressed oops. We "measure" them! So it also works with J9!

          Show
          Uwe Schindler added a comment - - edited Some restructuring and preventing a useless NPE that needs to be catched on non-Hotspot JVMs. I also removed the separate private method and made the whole thing with nested catches like at the other places in the static {} initializer (I don't like class' methods called from a static initializer). Also this one is easier to debug, as you can add printlns without working around the "return" escape of the original method! I tested on Windows: Oracle JDK 6u32 and 6u45 (64 bit) work, proprietary sunMF API used Oracle JDK 6u32, 32 bit is unsupported as before (the object alignment JVM option does not exist at all, but also the sunMF API is used) Oracle JDK 7u25 and Oracle JDK 8 preview (64 bit) works, new public Java 7 API is used (via reflection) or statically typed in trunk All other JVMs are unsupported, but they don't have obejct alignment available By the way: Our code looks better and more universal, because it does not rely on the management bean to detect compressed oops. We "measure" them! So it also works with J9!
          Hide
          Dawid Weiss added a comment -

          This looks good, thanks Uwe.

          Show
          Dawid Weiss added a comment - This looks good, thanks Uwe.
          Hide
          ASF subversion and git services added a comment -

          Commit 1499935 from Uwe Schindler
          [ https://svn.apache.org/r1499935 ]

          LUCENE-5086: RamUsageEstimator now uses official Java 7 API or a proprietary Oracle Java 6 API to get Hotspot MX bean, preventing AWT classes to be loaded on MacOSX

          Show
          ASF subversion and git services added a comment - Commit 1499935 from Uwe Schindler [ https://svn.apache.org/r1499935 ] LUCENE-5086 : RamUsageEstimator now uses official Java 7 API or a proprietary Oracle Java 6 API to get Hotspot MX bean, preventing AWT classes to be loaded on MacOSX
          Hide
          ASF subversion and git services added a comment -

          Commit 1499936 from Uwe Schindler
          [ https://svn.apache.org/r1499936 ]

          Merged revision(s) 1499935 from lucene/dev/trunk:
          LUCENE-5086: RamUsageEstimator now uses official Java 7 API or a proprietary Oracle Java 6 API to get Hotspot MX bean, preventing AWT classes to be loaded on MacOSX

          Show
          ASF subversion and git services added a comment - Commit 1499936 from Uwe Schindler [ https://svn.apache.org/r1499936 ] Merged revision(s) 1499935 from lucene/dev/trunk: LUCENE-5086 : RamUsageEstimator now uses official Java 7 API or a proprietary Oracle Java 6 API to get Hotspot MX bean, preventing AWT classes to be loaded on MacOSX
          Hide
          Uwe Schindler added a comment -

          Thanks Shay and Dawid!

          Show
          Uwe Schindler added a comment - Thanks Shay and Dawid!
          Hide
          Steve Rowe added a comment -

          Bulk close resolved 4.4 issues

          Show
          Steve Rowe added a comment - Bulk close resolved 4.4 issues

            People

            • Assignee:
              Uwe Schindler
              Reporter:
              Shay Banon
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development