Harmony
  1. Harmony
  2. HARMONY-4942

[classlib][jndi] InitialContext searches for jndi.properties every contruction time

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Classlib
    • Labels:
      None

      Description

      I'd like to bring attention of JNDI guys to the following issue.

      This is about InitialContext. Currently the InitialContext constructor implementation roughly speaking does the following:
      1. gets the properties passed to constructor (if any)
      2. searches the system properties for corresponding to JNDI stuff
      3. searches for jndi.properties in CLASSPATH and gets the properties from that (if any)

      (in case of InitialContext() we don't have the first step)

      This is quite cool until we face big amount of InitialContext() creations - then we see a lot of file system operations from step 3. For example during the Oracle App Server startup (with SPECjAppServer2004 onboard) - there is a lot of "new InitialContext()", every Bean creation. So we see about 27 millions of searches for "jndi.properties" in jar files from the Oracle App Server classpath (tens of jars).

      Is there anything can be done on this?..

      Well as it turned out the JDK1.6.0 DOES NOT searches for the "jndi.properties" in CLASSPATH for the second time we create InitialContext(). Once it's loaded the "jndi.properties" properties for the first time it does not bother to do it later. I suggest to do the same in harmony. Here is the output of the test demonstrating such a behaviour difference:

      {man@earth:~/tmp}

      cat jndi3.java
      import java.io.*;
      import javax.naming.*;

      public class jndi3 {
      public static void main(String args[]) throws Exception

      { createJndiPropertiesFile(); // create valid jndi.properties InitialContext context1 = new InitialContext(); System.out.println("context1= "+context1); System.out.println("context1.env="+context1.getEnvironment()); deleteJndiPropertiesFile(); // delete jndi.properties InitialContext context2 = new InitialContext(); System.out.println("context2= "+context2); System.out.println("context2.env="+context2.getEnvironment()); }

      public static void createJndiPropertiesFile() throws Exception

      { FileOutputStream fos = new FileOutputStream("jndi.properties"); PrintStream ps = new PrintStream(fos); ps.println("java.naming.factory.initial=MyDefaultInitialContextFactory"); ps.close(); }

      public static void deleteJndiPropertiesFile() throws Exception

      { File f = new File("jndi.properties"); f.delete(); }

      }

      {man@earth:~/tmp}

      /export/jdk1.6.0/bin/java jndi3
      context1= javax.naming.InitialContext@1034bb5
      context1.env={}
      context2= javax.naming.InitialContext@15f5897
      context2.env={}

      {man@earth:~/tmp}

      ~/harmony579850.clean/bin/java jndi3
      context1= javax.naming.InitialContext@108d9d1f
      context1.env={}
      context2= javax.naming.InitialContext@109242b5
      Uncaught exception in main:
      javax.naming.NoInitialContextException: Failed to create InitialContext using factory specified in hashtable {} [Root exception is java.lang.NullPointerException]
      at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:239)
      at javax.naming.InitialContext.initializeDefaultInitCtx(InitialContext.java:261)
      at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:291)
      at javax.naming.InitialContext.getEnvironment(InitialContext.java:540)
      at jndi3.main(jndi3.java:17)
      Caused by: java.lang.NullPointerException
      at java.lang.Class.forName(Unknown Source)
      at javax.naming.spi.NamingManager$1.run(NamingManager.java:772)
      at javax.naming.spi.NamingManager$1.run(NamingManager.java:1)
      at java.security.AccessController.doPrivilegedImpl(Unknown Source)
      at java.security.AccessController.doPrivileged(Unknown Source)
      at javax.naming.spi.NamingManager.classForName(NamingManager.java:768)
      at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:227)
      at javax.naming.InitialContext.initializeDefaultInitCtx(InitialContext.java:261)
      ... 3 more

      I've created a simple proof-of-concept of my idea: what about to not to do any of steps (1, 2, 3) not just 3? So please note that THIS IS NOT THE FIX! The actual fix should avoid only step 3 and probably should do it more carefully. Here are some measurements with respect to this (see jndi3per.java in attachment):

      {man@earth:~/tmp}

      ~/harmony579850.clean/bin/java jndi3perf 20000
      20000 constructor calls
      1. 17442 millis
      2. 11281 millis
      3. 11598 millis
      4. 10951 millis
      5. 10906 millis
      ^C

      {man@earth:~/tmp}

      ~/harmony579850.jndi3/bin/java jndi3perf 20000
      20000 constructor calls
      1. 466 millis
      2. 114 millis
      3. 107 millis
      4. 110 millis
      5. 107 millis
      ^C

      Talking back about Oracle App Server and SPECjAppServer2004: startup times burst comparing harmony without and with the proposed patch are:
      -Xem:client - 4X
      -Xem:server - 2x

      Furthermore SPECjAppServer2004 execution period is concerned to this as well: as it turned out we have ABOUT 3000 OF LOOKUPS of "jndi.properties" in jar files PER SECOND (transaction rate is about two hundreds). I'd say handling this should not decrease our current scores on SjAS2004.

      1. Harmony-4942.syn.fix.diff
        11 kB
        Regis Xu
      2. Harmony-4942.fix.diff
        10 kB
        Regis Xu
      3. Harmony-4942.new.diff
        11 kB
        Regis Xu
      4. Harmony-4942.new.diff
        11 kB
        Regis Xu
      5. Harmony-4942.diff
        10 kB
        Regis Xu
      6. jndi3_POC.patch
        1 kB
        Sergey Dmitriev
      7. jndi3perf.java
        1 kB
        Sergey Dmitriev
      8. jndi3.java
        1 kB
        Sergey Dmitriev
      9. MyInitialContext.java
        3 kB
        Sergey Dmitriev
      10. MyDefaultInitialContextFactory.java
        0.4 kB
        Sergey Dmitriev

        Activity

        Hide
        Regis Xu added a comment -

        Yes, the performance is very low, I agree that search the whole CLASSPATH should do one time. But this is a problems:
        we use thread context classloader (Thread.currentThread().getContextClassLoader()) to do the step 3, different classloaders
        may have different properties value. It very useful for app server which use different classloaders separate apps context. If just
        use static variable will cause conflict between classloaders. I'm not sure how ri do this, i will write test soon.

        Show
        Regis Xu added a comment - Yes, the performance is very low, I agree that search the whole CLASSPATH should do one time. But this is a problems: we use thread context classloader (Thread.currentThread().getContextClassLoader()) to do the step 3, different classloaders may have different properties value. It very useful for app server which use different classloaders separate apps context. If just use static variable will cause conflict between classloaders. I'm not sure how ri do this, i will write test soon.
        Hide
        Regis Xu added a comment -

        the path fix the problem that InitialContext search the whole CLASSPATH every construction time

        Show
        Regis Xu added a comment - the path fix the problem that InitialContext search the whole CLASSPATH every construction time
        Hide
        Tony Wu added a comment -

        Hi Regis,
        Seems the performance is not as good as ri. I did a quick look and found the bottleneck is that we call the file.exists() every time, basically, it is a very time-consuming operation with jni call. I suppose that ri just check the property file when vm init, would you please have a try on that?

        Show
        Tony Wu added a comment - Hi Regis, Seems the performance is not as good as ri. I did a quick look and found the bottleneck is that we call the file.exists() every time, basically, it is a very time-consuming operation with jni call. I suppose that ri just check the property file when vm init, would you please have a try on that?
        Hide
        Regis Xu added a comment -

        Hi Tony,
        I have test the ri's behavior, you are right, ri doesn't reload the property file under "java.home/lib/jndi.properties" after jvm init,
        the test prove it:

        ===================start====================
        Hashtable<Object, Object> env = new Hashtable<Object, Object>();
        env
        .put("java.naming.factory.initial",
        "org.apache.harmony.jndi.tests.javax.naming.spi.mock.MockContextFactory");

        InitialContext context = new InitialContext(env);

        System.out.println(context.getEnvironment().containsKey(
        "java.naming.provider.url"));

        // generate properties file
        File propsFile = new File(System.getProperty("java.home")
        + "/lib/jndi.properties");

        FileOutputStream fos = new FileOutputStream(propsFile);

        PrintStream ps = new PrintStream(fos);
        ps.println("java.naming.provider.url=http://test1");
        ps.close();

        context = new InitialContext(env);

        System.out.println(context.getEnvironment().containsKey(
        "java.naming.provider.url"));

        propsFile.delete();
        ========================end=================

        the output is:
        false
        false

        I will re-create a path to fix it soon.

        Show
        Regis Xu added a comment - Hi Tony, I have test the ri's behavior, you are right, ri doesn't reload the property file under "java.home/lib/jndi.properties" after jvm init, the test prove it: ===================start==================== Hashtable<Object, Object> env = new Hashtable<Object, Object>(); env .put("java.naming.factory.initial", "org.apache.harmony.jndi.tests.javax.naming.spi.mock.MockContextFactory"); InitialContext context = new InitialContext(env); System.out.println(context.getEnvironment().containsKey( "java.naming.provider.url")); // generate properties file File propsFile = new File(System.getProperty("java.home") + "/lib/jndi.properties"); FileOutputStream fos = new FileOutputStream(propsFile); PrintStream ps = new PrintStream(fos); ps.println("java.naming.provider.url= http://test1 "); ps.close(); context = new InitialContext(env); System.out.println(context.getEnvironment().containsKey( "java.naming.provider.url")); propsFile.delete(); ========================end================= the output is: false false I will re-create a path to fix it soon.
        Hide
        Regis Xu added a comment -

        Is there anyone to this?

        Show
        Regis Xu added a comment - Is there anyone to this?
        Hide
        Sergey Dmitriev added a comment -

        Regis I am watching this issue. Thank you for you work! Yesterday I've made performance measurements with respect to your first patch Harmony-4942.diff. Comparing to my POC version ( jndi3_POC.patch) your patch is 3% slower. But of course this is just for the record because my patch is not valid. The test I used is SPECjAppServer2004 on Oracle Application Server.

        I will post the measurement number of your second patch soon.

        Show
        Sergey Dmitriev added a comment - Regis I am watching this issue. Thank you for you work! Yesterday I've made performance measurements with respect to your first patch Harmony-4942.diff. Comparing to my POC version ( jndi3_POC.patch) your patch is 3% slower. But of course this is just for the record because my patch is not valid. The test I used is SPECjAppServer2004 on Oracle Application Server. I will post the measurement number of your second patch soon.
        Hide
        Regis Xu added a comment -

        Sergey, I test Harmony-4942.new.diff using your test case on my machine, the performance is still low than ri but have great improvement.
        the output is:
        on ri:
        20000 constructor calls (default)
        1. 167 millis
        2. 81 millis
        3. 82 millis
        4. 85 millis
        5. 86 millis
        6. 83 millis
        7. 82 millis
        8. 86 millis

        on harmony:
        20000 constructor calls (default)
        1. 653 millis
        2. 339 millis
        3. 316 millis
        4. 324 millis
        5. 317 millis
        6. 324 millis
        7. 329 millis
        8. 333 millis

        In this case, all properties files will be read only once. And because there is only one classloader, CLASSPATH will search once, i think
        there would be another bottleneck somewhere, i will try to find it.

        Show
        Regis Xu added a comment - Sergey, I test Harmony-4942.new.diff using your test case on my machine, the performance is still low than ri but have great improvement. the output is: on ri: 20000 constructor calls (default) 1. 167 millis 2. 81 millis 3. 82 millis 4. 85 millis 5. 86 millis 6. 83 millis 7. 82 millis 8. 86 millis on harmony: 20000 constructor calls (default) 1. 653 millis 2. 339 millis 3. 316 millis 4. 324 millis 5. 317 millis 6. 324 millis 7. 329 millis 8. 333 millis In this case, all properties files will be read only once. And because there is only one classloader, CLASSPATH will search once, i think there would be another bottleneck somewhere, i will try to find it.
        Hide
        Sergey Dmitriev added a comment -

        Regis, the numbers of your last improvement (11000 millis -> 300 millis) are really cool.

        Show
        Sergey Dmitriev added a comment - Regis, the numbers of your last improvement (11000 millis -> 300 millis) are really cool.
        Hide
        Regis Xu added a comment -

        last path doesn't fix file.exists problem, this one can fix it correctly, I think.
        And after this path on my machine, the performance of harmony is better than ri.

        Sergey, It's very kind of you and very helpful if you could use SPECjAppServer2004
        measure the path. Thanks.

        Show
        Regis Xu added a comment - last path doesn't fix file.exists problem, this one can fix it correctly, I think. And after this path on my machine, the performance of harmony is better than ri. Sergey, It's very kind of you and very helpful if you could use SPECjAppServer2004 measure the path. Thanks.
        Hide
        Sergey Dmitriev added a comment -

        Regis thanks for you fix! Unfortunately due to technical reasons I cannot check the performance of your fix. I just can say that your patch works. I'll try to check the performance as soon as it will be possible.

        I looked at the diff and and just caught you're using HashMap: don't you think we can have a sync issues because of using HashMap not Hashtable as a map [classloader -> props]?

        Show
        Sergey Dmitriev added a comment - Regis thanks for you fix! Unfortunately due to technical reasons I cannot check the performance of your fix. I just can say that your patch works. I'll try to check the performance as soon as it will be possible. I looked at the diff and and just caught you're using HashMap: don't you think we can have a sync issues because of using HashMap not Hashtable as a map [classloader -> props] ?
        Hide
        Regis Xu added a comment -

        Sergey thanks for your reminder! I agree the sync issues is necessary, I will fix it soon.
        Thanks again for your helps!

        Show
        Regis Xu added a comment - Sergey thanks for your reminder! I agree the sync issues is necessary, I will fix it soon. Thanks again for your helps!
        Hide
        Regis Xu added a comment -

        this patch(Harmony-4942.syn.fix.diff) use Hashtable instead of HashMap for propsCache to
        make it thread safe.

        Show
        Regis Xu added a comment - this patch(Harmony-4942.syn.fix.diff) use Hashtable instead of HashMap for propsCache to make it thread safe.

          People

          • Assignee:
            Tony Wu
            Reporter:
            Sergey Dmitriev
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:

              Development