Derby
  1. Derby
  2. DERBY-2469

Java Web Start JNLP PersistenceService API storage support

    Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: 10.2.2.0
    • Fix Version/s: None
    • Component/s: Store
    • Labels:
      None
    • Environment:
      Java Web Start
    • Urgency:
      Low

      Description

      I would love to have Derby write/read to the storage area provided by the JNLP PersistenceService API.

      Since Derby is now bundled with the Java6 JDK as JavaDB, I think this integration would go a long way towards making derby more developer- friendly in Java Web Start environments, where using the sandbox tools Sun provides us it the right way to go, instead of working around it and force the user to give the app the authorization to write on the hard drive IMHO.

      I'm investigating the effort needed to provide an implementation of the WritableStorageFactory interface around the PersistenceService API, and if that's doable in a few days work, I will start working on it and submit a patch for testing/approval ASAP.

      Feel free to volounteer and provide pointers/hints/whatever, it's really appreciate, especially since I currently know nothing of derby internals.

      1. svn-diff-20070329
        42 kB
        Luigi Lauro
      2. svn-diff-20070606
        46 kB
        Luigi Lauro
      3. svn-diff-20070612
        46 kB
        Luigi Lauro
      4. svn-diff-20070626
        52 kB
        David Van Couvering
      5. jnlptest.jar
        14 kB
        David Van Couvering

        Activity

        Hide
        David Van Couvering added a comment -

        I think Francois is working on this now...

        Show
        David Van Couvering added a comment - I think Francois is working on this now...
        Hide
        David Van Couvering added a comment -

        This is a NetBeans project (using NetBeans 6) that attempts to use the JNLP storage provider and it fails (see exception below). I'm not sure if the failure is because of no suitable driver or because the attempt to ready derby.log hit an AccessControlException, which then caused the no suitable driver exception (I'm suspecting the latter).

        You can build this independent of NetBeans using Ant, but if you want to test, I highly recommend getting the latest milestone of NetBeans 6,
        which can be found at

        http://dlc.sun.com/netbeans/download/6.0/milestones/latest/

        Error messages:

        2007-06-26 16:59:33.189 java[1746] CFLog (0): CFMessagePort: bootstrap_register(): failed 1103 (0x44f), port = 0xf503, name = 'java.ServiceProvider'
        See /usr/include/servers/bootstrap_defs.h for the error codes.
        2007-06-26 16:59:33.190 java[1746] CFLog (99): CFMessagePortCreateLocal(): failed to name Mach port (java.ServiceProvider)
        2007-06-26 23:59:36.986 GMT Thread[javawsApplicationMain,5,javawsApplicationThreadGroup] java.security.AccessControlException: access denied (java.io.FilePermission derby.log read)
        java.sql.SQLException: No suitable driver
        at java.sql.DriverManager.getConnection(DriverManager.java:545)
        at java.sql.DriverManager.getConnection(DriverManager.java:193)
        at testjnlp2.Main.main(Main.java:34)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:585)
        at com.sun.javaws.Launcher.executeApplication(Launcher.java:1161)
        at com.sun.javaws.Launcher.executeMainClass(Launcher.java:1108)
        at com.sun.javaws.Launcher.continueLaunch(Launcher.java:951)
        at com.sun.javaws.Launcher.handleApplicationDesc(Launcher.java:522)
        at com.sun.javaws.Launcher.handleLaunchFile(Launcher.java:218)
        at com.sun.javaws.Launcher.run(Launcher.java:165)
        at java.lang.Thread.run(Thread.java:613)

        Show
        David Van Couvering added a comment - This is a NetBeans project (using NetBeans 6) that attempts to use the JNLP storage provider and it fails (see exception below). I'm not sure if the failure is because of no suitable driver or because the attempt to ready derby.log hit an AccessControlException, which then caused the no suitable driver exception (I'm suspecting the latter). You can build this independent of NetBeans using Ant, but if you want to test, I highly recommend getting the latest milestone of NetBeans 6, which can be found at http://dlc.sun.com/netbeans/download/6.0/milestones/latest/ Error messages: 2007-06-26 16:59:33.189 java [1746] CFLog (0): CFMessagePort: bootstrap_register(): failed 1103 (0x44f), port = 0xf503, name = 'java.ServiceProvider' See /usr/include/servers/bootstrap_defs.h for the error codes. 2007-06-26 16:59:33.190 java [1746] CFLog (99): CFMessagePortCreateLocal(): failed to name Mach port (java.ServiceProvider) 2007-06-26 23:59:36.986 GMT Thread [javawsApplicationMain,5,javawsApplicationThreadGroup] java.security.AccessControlException: access denied (java.io.FilePermission derby.log read) java.sql.SQLException: No suitable driver at java.sql.DriverManager.getConnection(DriverManager.java:545) at java.sql.DriverManager.getConnection(DriverManager.java:193) at testjnlp2.Main.main(Main.java:34) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at com.sun.javaws.Launcher.executeApplication(Launcher.java:1161) at com.sun.javaws.Launcher.executeMainClass(Launcher.java:1108) at com.sun.javaws.Launcher.continueLaunch(Launcher.java:951) at com.sun.javaws.Launcher.handleApplicationDesc(Launcher.java:522) at com.sun.javaws.Launcher.handleLaunchFile(Launcher.java:218) at com.sun.javaws.Launcher.run(Launcher.java:165) at java.lang.Thread.run(Thread.java:613)
        Hide
        David Van Couvering added a comment -

        Here is an updated patch that builds within Derby. You need to build this with JDK6 (see BUILDING.txt for how to enable this).

        I don't think this actually works yet, but at least it builds. I am hoping this can be used by others to move forward on testing.

        Show
        David Van Couvering added a comment - Here is an updated patch that builds within Derby. You need to build this with JDK6 (see BUILDING.txt for how to enable this). I don't think this actually works yet, but at least it builds. I am hoping this can be used by others to move forward on testing.
        Hide
        Luigi Lauro added a comment -

        Any news guys? I'm looking forward to some feedback if possible

        Thanks again,

        Show
        Luigi Lauro added a comment - Any news guys? I'm looking forward to some feedback if possible Thanks again,
        Hide
        David Van Couvering added a comment -

        Dropping from 10.3, it's just too tight. I will work on getting it checked into the trunk after the 10.3 release.

        Show
        David Van Couvering added a comment - Dropping from 10.3, it's just too tight. I will work on getting it checked into the trunk after the 10.3 release.
        Hide
        Lorenzo Sicilia added a comment -

        I think this feature is strategic for derby and java in general.
        Google gear (http://www.pcworld.in/news/index.jsp/artId=5668922) and adobe apollo (http://www.mikechambers.com/blog/2007/05/30/apollo-beta-will-include-sqlite-embedded-database/) work to get sqlite as database includes in his application.
        Java at the moment have good database (derby) but without support of java web start. it's a really issue.
        I hope we can get soon this feature.
        Just my 2 cent.

        Show
        Lorenzo Sicilia added a comment - I think this feature is strategic for derby and java in general. Google gear ( http://www.pcworld.in/news/index.jsp/artId=5668922 ) and adobe apollo ( http://www.mikechambers.com/blog/2007/05/30/apollo-beta-will-include-sqlite-embedded-database/ ) work to get sqlite as database includes in his application. Java at the moment have good database (derby) but without support of java web start. it's a really issue. I hope we can get soon this feature. Just my 2 cent.
        Hide
        Luigi Lauro added a comment -

        Here my answers:

        @Mike: Ok, sorry man, from next patch on, the paths will be relative to derby root, don't worry My mistake, is that I just checked out what I needed and not the whole tree _

        Also, regarding jdk1.4: I will look into it once the feature is complete and stable. A backporting to 1.4 should be really easy to do, but I prefer to stick with 1.5 in the meanwhile, for easiness of coding (for each, generics, etc...).

        @Kathey: you got me completely wrong. I don't think a MATURE STABLE 10.3 'official feature' JNLPStorageFactory is anytime near. I won't ever think such a delicate feature (storage) would ever go into such a quality product as derby, without a long and deep QA.

        What I meant is that I would really love some help with the testing/code review here, to get feedback and see what I may have done wrong (I HAVE surely done something wrong, given my little derby experience, even being a quite experienced java programmer as I am). If 'checking it in' as "EXPERIMENTAL - TURN IT ON AND YOU DIE" could make more ppl willing to give it a try and provide feedback, then you have my vote for 10.3 experimental inclusion.

        But of course, any choice will be fine for me

        @David: Man, I love your plan And btw, org.apache.derby.impl.io.jnlp all the way, given where the other StorageFactory implementations reside currently.

        Regarding the unit testing I did for my classes: nothing 'givable', mostly manual unit testing for checking the correctness of the most obvious features.

        I was planning to add a full JUnit unit testing once I got more knowledge about the StorageFactory internals: I still have some doubts if I got it right by reading the other factories and checking javadocs, and becuase of that, I could have coded wrong tests with wrong assumptions, by doing them before even knowing exactly HOW each method should perfectly behave (the javadocs are NOT entirely clear on these matters, and I had to dwelve into the code to check my assumptions).

        I heard there is a 'storage' junit test suite, and that would probably be the first thing to test out once we get it running past the first obvious mistakes that will come out.

        One final note: sorry guys for asking for help on this, I know it would have better to work on it 24/7 to have a more 'mature' feature and then submit for reviewal, but given my little derby experience I would have lost a lot more time this way, and my employer doesn't want me to spend too much time on this unluckily (other things have priority right now).

        Show
        Luigi Lauro added a comment - Here my answers: @Mike: Ok, sorry man, from next patch on, the paths will be relative to derby root, don't worry My mistake, is that I just checked out what I needed and not the whole tree _ Also, regarding jdk1.4: I will look into it once the feature is complete and stable. A backporting to 1.4 should be really easy to do, but I prefer to stick with 1.5 in the meanwhile, for easiness of coding (for each, generics, etc...). @Kathey: you got me completely wrong. I don't think a MATURE STABLE 10.3 'official feature' JNLPStorageFactory is anytime near. I won't ever think such a delicate feature (storage) would ever go into such a quality product as derby, without a long and deep QA. What I meant is that I would really love some help with the testing/code review here, to get feedback and see what I may have done wrong (I HAVE surely done something wrong, given my little derby experience, even being a quite experienced java programmer as I am). If 'checking it in' as "EXPERIMENTAL - TURN IT ON AND YOU DIE" could make more ppl willing to give it a try and provide feedback, then you have my vote for 10.3 experimental inclusion. But of course, any choice will be fine for me @David: Man, I love your plan And btw, org.apache.derby.impl.io.jnlp all the way, given where the other StorageFactory implementations reside currently. Regarding the unit testing I did for my classes: nothing 'givable', mostly manual unit testing for checking the correctness of the most obvious features. I was planning to add a full JUnit unit testing once I got more knowledge about the StorageFactory internals: I still have some doubts if I got it right by reading the other factories and checking javadocs, and becuase of that, I could have coded wrong tests with wrong assumptions, by doing them before even knowing exactly HOW each method should perfectly behave (the javadocs are NOT entirely clear on these matters, and I had to dwelve into the code to check my assumptions). I heard there is a 'storage' junit test suite, and that would probably be the first thing to test out once we get it running past the first obvious mistakes that will come out. One final note: sorry guys for asking for help on this, I know it would have better to work on it 24/7 to have a more 'mature' feature and then submit for reviewal, but given my little derby experience I would have lost a lot more time this way, and my employer doesn't want me to spend too much time on this unluckily (other things have priority right now).
        Hide
        David Van Couvering added a comment -

        OK, here is my proposal:

        • Let's not try to shove it into this release
        • Let's get this checked in very soon so people can start playing with it.
        • What I would like is if our nightly builds are easily available off of the web. How hard is this to do? But that's a separate discussion, and I'll re-ask this question on derby-dev
        • I'll put it into the experimental directory and have it build for JDK 1.5 or above, into derby-experimental.jar
        • Should the package name be org.apache.derby.impl.io.jnlp or org.apache.derby.jnlp? I'm leaning for the former, as it is an IO implementation.
        • I woud like to check this in (after Friday) so that it builds, but without unit tests. That way Luigi and I and others can collaborate better on adding appropriate testing
        • The goal is that hopefully once this is seen to be working and useful, we can migrate the package from the experimental tree to the main tree (and into derby.jar), and integrate the unit tests into the full regression suite.

        Does this sound like a plan?

        Luigi, I have a fair idea how to run this within Derby, but I'm still testing this out. What I want to know is how you tested it at all? How did you test this outside of Derby? If you have JUnit unit tests for this, let's have them, being able to run unit tests for these classes independent of Derby IMHO is a Good Thing. You can ask the list where the unit tests should go, I'm not too clear on that myself, but at a minimum just post them as a jar file to this JIRA so we can have a look at them.

        David

        Show
        David Van Couvering added a comment - OK, here is my proposal: Let's not try to shove it into this release Let's get this checked in very soon so people can start playing with it. What I would like is if our nightly builds are easily available off of the web. How hard is this to do? But that's a separate discussion, and I'll re-ask this question on derby-dev I'll put it into the experimental directory and have it build for JDK 1.5 or above, into derby-experimental.jar Should the package name be org.apache.derby.impl.io.jnlp or org.apache.derby.jnlp? I'm leaning for the former, as it is an IO implementation. I woud like to check this in (after Friday) so that it builds, but without unit tests. That way Luigi and I and others can collaborate better on adding appropriate testing The goal is that hopefully once this is seen to be working and useful, we can migrate the package from the experimental tree to the main tree (and into derby.jar), and integrate the unit tests into the full regression suite. Does this sound like a plan? Luigi, I have a fair idea how to run this within Derby, but I'm still testing this out. What I want to know is how you tested it at all? How did you test this outside of Derby? If you have JUnit unit tests for this, let's have them, being able to run unit tests for these classes independent of Derby IMHO is a Good Thing. You can ask the list where the unit tests should go, I'm not too clear on that myself, but at a minimum just post them as a jar file to this JIRA so we can have a look at them. David
        Hide
        Kathey Marsden added a comment -

        My opinion from Luigi's comment:
        >I Consider the implementation feature-complete 95%, but I HAVE NOT DONE ANY TEST WITH IT YET. Only unit >testing, to catch the most evident bugs/typos, but NO INTEGRATION TESTING at all, presently. I think it will fail >horribly at start because of several reasons I could have foreseen given my little derby experience
        ....

        >I would also appreciate a lot code review with someone with deep knowledge about StorageFactory/StorageFile interfaces and how should they behave

        That this sounds like it is more than a day away from ready to go in. Just my 2c.

        Kathey

        Show
        Kathey Marsden added a comment - My opinion from Luigi's comment: >I Consider the implementation feature-complete 95%, but I HAVE NOT DONE ANY TEST WITH IT YET. Only unit >testing, to catch the most evident bugs/typos, but NO INTEGRATION TESTING at all, presently. I think it will fail >horribly at start because of several reasons I could have foreseen given my little derby experience .... >I would also appreciate a lot code review with someone with deep knowledge about StorageFactory/StorageFile interfaces and how should they behave That this sounds like it is more than a day away from ready to go in. Just my 2c. Kathey
        Hide
        Mike Matrigali added a comment -

        The suggestions by david for handling the experimental checkin sounds reasonable to me. I still think this code is a too early for a release, given no testing and todo's like "check if synchronization is needed", but I will leave the choice up to myrna. The problem
        I see is that if it gets into the release it seems very likely more changes will be necessary pretty soon, and then you have lost the
        testing audience as they won't be testing what you want. You want a more frequent turnaround for this, which I think would be
        better served by decoupling the experimental jars from a "supported" release. Getting it into the development trunk is the first
        step so that at least derby developers can always easily get up to date versions of the code. I am not as sure how to provide
        derby user community with very quick up to date experimental jars, I think the requirements for this are very different than the
        standard code.

        Will the testing that is being done currently result in new junit tests be added to the nightly suites?

        Show
        Mike Matrigali added a comment - The suggestions by david for handling the experimental checkin sounds reasonable to me. I still think this code is a too early for a release, given no testing and todo's like "check if synchronization is needed", but I will leave the choice up to myrna. The problem I see is that if it gets into the release it seems very likely more changes will be necessary pretty soon, and then you have lost the testing audience as they won't be testing what you want. You want a more frequent turnaround for this, which I think would be better served by decoupling the experimental jars from a "supported" release. Getting it into the development trunk is the first step so that at least derby developers can always easily get up to date versions of the code. I am not as sure how to provide derby user community with very quick up to date experimental jars, I think the requirements for this are very different than the standard code. Will the testing that is being done currently result in new junit tests be added to the nightly suites?
        Hide
        Knut Anders Hatlen added a comment -

        The following worked for me (with GNU patch):

        $ cd java/engine
        $ gpatch -p1 < ../../svn-diff-20070612
        patching file org/apache/derby/impl/io/AbstractStorageFactory.java
        patching file org/apache/derby/impl/io/JNLPStorageFactory.java
        patching file org/apache/derby/impl/io/JNLPStorageFile.java

        Show
        Knut Anders Hatlen added a comment - The following worked for me (with GNU patch): $ cd java/engine $ gpatch -p1 < ../../svn-diff-20070612 patching file org/apache/derby/impl/io/AbstractStorageFactory.java patching file org/apache/derby/impl/io/JNLPStorageFactory.java patching file org/apache/derby/impl/io/JNLPStorageFile.java
        Hide
        Mike Matrigali added a comment -

        It is up to you if you want to make the code buildable under 1.4. I think this will allow for a wider audience, especially if there are
        no real dependencies on post 1.4 features. All that I am looking for is that any checkin, do the appropriate thing to the ant build
        files so that if a developer only has jdk1.4 that the system won't try to build this code. I believe the current requirement for trunk
        and 10.3 is that the system should build cleanly and run tests cleanly for jdk1.4..2 jvm's. It may not build all classes, and may not
        run all code (ie. jdbc4.0) testing.

        I believe the following shows build/tests for both building against jdk1.4/running various jvm's and building against 1.6 :
        http://dbtg.thresher.com/derby/test/

        Once the build changes are made, I would be happy to try out the patch and verify it does not cause build errors in a 1.4 only system,
        as that is the environment I tend to run in.

        Show
        Mike Matrigali added a comment - It is up to you if you want to make the code buildable under 1.4. I think this will allow for a wider audience, especially if there are no real dependencies on post 1.4 features. All that I am looking for is that any checkin, do the appropriate thing to the ant build files so that if a developer only has jdk1.4 that the system won't try to build this code. I believe the current requirement for trunk and 10.3 is that the system should build cleanly and run tests cleanly for jdk1.4..2 jvm's. It may not build all classes, and may not run all code (ie. jdbc4.0) testing. I believe the following shows build/tests for both building against jdk1.4/running various jvm's and building against 1.6 : http://dbtg.thresher.com/derby/test/ Once the build changes are made, I would be happy to try out the patch and verify it does not cause build errors in a 1.4 only system, as that is the environment I tend to run in.
        Hide
        Mike Matrigali added a comment -

        The right kind of relative path I would expect in a patch would be something like:
        Index: java/engine/org/apache/derby/impl/io/JNLPStorageFactory.java

        I use a particularly finicky version of patch, if someone else is sucessfully applying the patches included let me know. I expect
        patches to be relative to the top of the source tree, ie. the opensource directory.

        Show
        Mike Matrigali added a comment - The right kind of relative path I would expect in a patch would be something like: Index: java/engine/org/apache/derby/impl/io/JNLPStorageFactory.java I use a particularly finicky version of patch, if someone else is sucessfully applying the patches included let me know. I expect patches to be relative to the top of the source tree, ie. the opensource directory.
        Hide
        Luigi Lauro added a comment -

        @Myrna/David regarding target 10.3: you decide guys, my focus is to get this feature done and working in the least time possible. If putting it as experimental in 10.3 to allow more ppl to test it up after we did some initial tests with David & CO is fine, then ok, otherwise np, we will wait for 10.4 for having this official in derby

        @David regarding enabling this feature:

        David, maybe you lost some of my comments, so I repeat myself again.

        I Consider the implementation feature-complete 95%, but I HAVE NOT DONE ANY TEST WITH IT YET. Only unit testing, to catch the most evident bugs/typos, but NO INTEGRATION TESTING at all, presently. I think it will fail horribly at start because of several reasons I could have foreseen given my little derby experience, but since the implementation is very clean and straightforward, I'm positive that I'll be able to sort out things and make it work for real in small time.

        Presently, what I need is someone with knowledge of derby internal to tell me HOW to enable this storagefactory with derby and test it, and maybe help me with the testing as well and provide feedback, so that I can fix.

        I would also appreciate a lot code review with someone with deep knowledge about StorageFactory/StorageFile interfaces and how should they behave, would also really be appreciated, to get things I may have got wrong (javadocs were not ENTIRELY clear about certain matters, and I had to 'derive' some things from the code itself of the other factories). This can be easily done, my code is really easy to read and understand (check and see).

        @Mike regarding SVN-DIFF: AFAIK the latest svn patch (https://issues.apache.org/jira/secure/attachment/12359502/svn-diff-20070612) should have right relative paths (example: — /org/apache/derby/impl/io/AbstractStorageFactory.java (revision 0)) and all the code should be JVM1.5 compliant (I removed the 1.6 stuff). Later, I could also backport it to 1.4 (easily done), mostly by removing generics.

        So please, if something is wrong with the patch let me know, so I can fix it and provide 'working' patches in next build.

        Show
        Luigi Lauro added a comment - @Myrna/David regarding target 10.3: you decide guys, my focus is to get this feature done and working in the least time possible. If putting it as experimental in 10.3 to allow more ppl to test it up after we did some initial tests with David & CO is fine, then ok, otherwise np, we will wait for 10.4 for having this official in derby @David regarding enabling this feature: David, maybe you lost some of my comments, so I repeat myself again. I Consider the implementation feature-complete 95%, but I HAVE NOT DONE ANY TEST WITH IT YET. Only unit testing, to catch the most evident bugs/typos, but NO INTEGRATION TESTING at all, presently. I think it will fail horribly at start because of several reasons I could have foreseen given my little derby experience, but since the implementation is very clean and straightforward, I'm positive that I'll be able to sort out things and make it work for real in small time. Presently, what I need is someone with knowledge of derby internal to tell me HOW to enable this storagefactory with derby and test it, and maybe help me with the testing as well and provide feedback, so that I can fix. I would also appreciate a lot code review with someone with deep knowledge about StorageFactory/StorageFile interfaces and how should they behave, would also really be appreciated, to get things I may have got wrong (javadocs were not ENTIRELY clear about certain matters, and I had to 'derive' some things from the code itself of the other factories). This can be easily done, my code is really easy to read and understand (check and see). @Mike regarding SVN-DIFF: AFAIK the latest svn patch ( https://issues.apache.org/jira/secure/attachment/12359502/svn-diff-20070612 ) should have right relative paths (example: — /org/apache/derby/impl/io/AbstractStorageFactory.java (revision 0)) and all the code should be JVM1.5 compliant (I removed the 1.6 stuff). Later, I could also backport it to 1.4 (easily done), mostly by removing generics. So please, if something is wrong with the patch let me know, so I can fix it and provide 'working' patches in next build.
        Hide
        David Van Couvering added a comment -

        Tentatively making this fix-in 10.3, understanding that if we don't find a solution Myrna will un-check this.

        Show
        David Van Couvering added a comment - Tentatively making this fix-in 10.3, understanding that if we don't find a solution Myrna will un-check this.
        Hide
        David Van Couvering added a comment -

        I agree it needs to get tested, and I'm working on that (and I've asked MIchelle to help)

        But yes I believe it should be considered experimental. I like the idea of putting it into a separate jar file.

        How about the following approach:

        • Create a new top-level subdirectory called 'experimental'
        • Create a package org.apache.derby.jnlp and put the new classes there (fixing them as necessary so they do not require package-private access to the default implementation)
        • build-jars creates a new jar file, derby-experimental.jar, that contains all the classes in the experimental subdirectory

        The way I think you use it (I haven't been able to verify this actually works yet) is that you add a new subsubProtocol in modules.properties called 'jnlp' (derby.storage.subsubProtocol.jnlp6=org.apache.derby.jnlp.JNLPStorageFactory) and then use the URL "jdbc:derby:jnlp:mydb;create=true" (or something to that effect). You would add this to modules.properties so it is only enabled for Java 5 or higher.

        The intended use is that when you create a Java Web Start application, you use this storage engine, and this makes it so the JWS application doesn't have to ask the user to grant permission to use the local file system.

        Woudl this be acceptable for checkin?

        I also want to get it tested, and will work on that, but I would think that if it doesn't break the build and doesn't go into the standard derby.jar, it should be check-inable without tests. This is similar to the way we treat our demo directory, it seems to me. We want to be able to check in stuff like this without having to go through a big gauntlet. We almost Luigi on this one, and we still don't have the in-memory storage and JMX support checked in (both of which could I am pretty sure could be put in the experimental jar).

        David

        Show
        David Van Couvering added a comment - I agree it needs to get tested, and I'm working on that (and I've asked MIchelle to help) But yes I believe it should be considered experimental. I like the idea of putting it into a separate jar file. How about the following approach: Create a new top-level subdirectory called 'experimental' Create a package org.apache.derby.jnlp and put the new classes there (fixing them as necessary so they do not require package-private access to the default implementation) build-jars creates a new jar file, derby-experimental.jar, that contains all the classes in the experimental subdirectory The way I think you use it (I haven't been able to verify this actually works yet) is that you add a new subsubProtocol in modules.properties called 'jnlp' (derby.storage.subsubProtocol.jnlp6=org.apache.derby.jnlp.JNLPStorageFactory) and then use the URL "jdbc:derby:jnlp:mydb;create=true" (or something to that effect). You would add this to modules.properties so it is only enabled for Java 5 or higher. The intended use is that when you create a Java Web Start application, you use this storage engine, and this makes it so the JWS application doesn't have to ask the user to grant permission to use the local file system. Woudl this be acceptable for checkin? I also want to get it tested, and will work on that, but I would think that if it doesn't break the build and doesn't go into the standard derby.jar, it should be check-inable without tests. This is similar to the way we treat our demo directory, it seems to me. We want to be able to check in stuff like this without having to go through a big gauntlet. We almost Luigi on this one, and we still don't have the in-memory storage and JMX support checked in (both of which could I am pretty sure could be put in the experimental jar). David
        Hide
        Mike Matrigali added a comment -

        I hacked the diff file to put the new files where I think it wanted them. If this gets checked in as is, then it will create build errors for those building with jdk1.4.

        Before getting checked in, some build info needs to get put into the ant xml files so that these files only get built with 1.6 vs. 1.4. This magic currently exists for all the jdbc4.0 support that went in. I am not sure exactly what need to change there, maybe someone who did that work can point a finger.

        Show
        Mike Matrigali added a comment - I hacked the diff file to put the new files where I think it wanted them. If this gets checked in as is, then it will create build errors for those building with jdk1.4. Before getting checked in, some build info needs to get put into the ant xml files so that these files only get built with 1.6 vs. 1.4. This magic currently exists for all the jdbc4.0 support that went in. I am not sure exactly what need to change there, maybe someone who did that work can point a finger.
        Hide
        Mike Matrigali added a comment -

        I am concerned about making this part of the release that is coming up. I haven't been able to apply the patch, but as far as I can tell there is no test to verify it works at all. Also not clear what needs to be done to use it.

        I do wish Derby had a better mechanism for delivering experimental code to users who would rather not apply patches and build for themselves. Until this module is tested it seems like it should not be part of a delivered derby jar.

        Derby has been built to allow for just these kinds of extensions, it would be nice if we could somehow improve the release mechanism to allow for this, but whether myrna wants to do that is up to her. I would like it if these extensions could be added to a separate jar, maybe derby_experimental.jar to make it very clear that it is not currently supported. I havent seen much write up for this feature so I don't know if it might affect backward compatibility, or somehow affect files in existing databases if called on them. We may want to log warnings to errror logs about experimental, may want to mark dbs as beta, ....

        I think it would also be nice if there was an area of the trunk where things like
        this could get checked in so that multiple people could work on stuff like this but
        where it was easy for a release manager could pick and choose whether it
        should be included in an actual release. In this case it might also serve to
        make sure that the interfaces are right for this module such that one need
        not bundle the new module with the default implementation.

        Show
        Mike Matrigali added a comment - I am concerned about making this part of the release that is coming up. I haven't been able to apply the patch, but as far as I can tell there is no test to verify it works at all. Also not clear what needs to be done to use it. I do wish Derby had a better mechanism for delivering experimental code to users who would rather not apply patches and build for themselves. Until this module is tested it seems like it should not be part of a delivered derby jar. Derby has been built to allow for just these kinds of extensions, it would be nice if we could somehow improve the release mechanism to allow for this, but whether myrna wants to do that is up to her. I would like it if these extensions could be added to a separate jar, maybe derby_experimental.jar to make it very clear that it is not currently supported. I havent seen much write up for this feature so I don't know if it might affect backward compatibility, or somehow affect files in existing databases if called on them. We may want to log warnings to errror logs about experimental, may want to mark dbs as beta, .... I think it would also be nice if there was an area of the trunk where things like this could get checked in so that multiple people could work on stuff like this but where it was easy for a release manager could pick and choose whether it should be included in an actual release. In this case it might also serve to make sure that the interfaces are right for this module such that one need not bundle the new module with the default implementation.
        Hide
        Mike Matrigali added a comment -

        Is there a trick to applying the latest patch, I think the paths are wrong?

        Show
        Mike Matrigali added a comment - Is there a trick to applying the latest patch, I think the paths are wrong?
        Hide
        David Van Couvering added a comment -

        Myrna: I don't understand why this can't go into the release if it's not going to be used unless explicitly enabled by a user. Isn't there a way we can mark this as experimental? It's not like it's going to regress existing functionality.

        David

        Show
        David Van Couvering added a comment - Myrna: I don't understand why this can't go into the release if it's not going to be used unless explicitly enabled by a user. Isn't there a way we can mark this as experimental? It's not like it's going to regress existing functionality. David
        Hide
        David Van Couvering added a comment -

        Rats, somehow I'm missing these JIRA comments, I'll fix my filters.

        Luigi, thanks for the new patch, I'll try it out. Meantime, I did the same thing, so I'll compare and contrast.

        I tried running my version under JWS and I got this error

        2007-06-12 19:11:59.936 GMT Thread[javawsApplicationMain,5,javawsApplicationThreadGroup] java.security.AccessControlException: access denied (java.io.FilePermission derby.log read)

        Do you get this?

        Also, I'm not sure what you have done to enable Derby to use this storage factory, since you haven't added an entry for it in modules.properties (at least you didn't in your last patch). How do you actually run and test this???

        David

        Show
        David Van Couvering added a comment - Rats, somehow I'm missing these JIRA comments, I'll fix my filters. Luigi, thanks for the new patch, I'll try it out. Meantime, I did the same thing, so I'll compare and contrast. I tried running my version under JWS and I got this error 2007-06-12 19:11:59.936 GMT Thread [javawsApplicationMain,5,javawsApplicationThreadGroup] java.security.AccessControlException: access denied (java.io.FilePermission derby.log read) Do you get this? Also, I'm not sure what you have done to enable Derby to use this storage factory, since you haven't added an entry for it in modules.properties (at least you didn't in your last patch). How do you actually run and test this??? David
        Hide
        Luigi Lauro added a comment -

        Updated version.

        1) Fixed a problem with deleteAll() and implemented renameTo(), and cleaned a couple of methods.

        2) Made it JVM 1.5 compliant, by removing the unneeded String.isEmpty and new IOException(String, Throwable).

        3) Checked against latest version of SVN, and with correct relative paths.

        Enjoy

        Show
        Luigi Lauro added a comment - Updated version. 1) Fixed a problem with deleteAll() and implemented renameTo(), and cleaned a couple of methods. 2) Made it JVM 1.5 compliant, by removing the unneeded String.isEmpty and new IOException(String, Throwable). 3) Checked against latest version of SVN, and with correct relative paths. Enjoy
        Hide
        Luigi Lauro added a comment -

        Ok, my employer told me he correctly just sent a CLA for Kemen at Apache Foundation, including myself as employee, so you know have my ICLA for the JNLPStorageFactory implementation.

        Show
        Luigi Lauro added a comment - Ok, my employer told me he correctly just sent a CLA for Kemen at Apache Foundation, including myself as employee, so you know have my ICLA for the JNLPStorageFactory implementation.
        Hide
        Luigi Lauro added a comment -

        Myrna:

        The implementation is considered nearly feature-complete: only testing is need to be done, but I think it will be pretty straightforward (the implementation is simple and clean enough to sort problems quickly and easily I think).

        The real work will come from improving it (linear time lookups for entities by using indexing structures, in-memory temporary files, etc...), but a working implementaiton is not far away IMHO.

        Also, you will see an CLA very soon from Kemen, the company which I'm working for. Later today, to be precise.

        I would really like to see this make into the 10.3 time frame, especially since it's an 'addon' to derby, and even if working so and so, would not hinder derby in any way unless someone explicitly know about this and enable this StorageFactory, but in the meanwhile we could have a lot more ppl test it and provide feedback to speed up the implementation fixing and polishing.

        My 2c.

        Show
        Luigi Lauro added a comment - Myrna: The implementation is considered nearly feature-complete: only testing is need to be done, but I think it will be pretty straightforward (the implementation is simple and clean enough to sort problems quickly and easily I think). The real work will come from improving it (linear time lookups for entities by using indexing structures, in-memory temporary files, etc...), but a working implementaiton is not far away IMHO. Also, you will see an CLA very soon from Kemen, the company which I'm working for. Later today, to be precise. I would really like to see this make into the 10.3 time frame, especially since it's an 'addon' to derby, and even if working so and so, would not hinder derby in any way unless someone explicitly know about this and enable this StorageFactory, but in the meanwhile we could have a lot more ppl test it and provide feedback to speed up the implementation fixing and polishing. My 2c.
        Hide
        Myrna van Lunteren added a comment -

        I am unsetting this for 10.3 for the following reasons:

        • It seems there's quite some work left to do,
        • I'd consider this a contribution that would need an ICLA and I've not seen one
        • I think we should evaluate where this will end up - in derby.jar or somewhere else?
        Show
        Myrna van Lunteren added a comment - I am unsetting this for 10.3 for the following reasons: It seems there's quite some work left to do, I'd consider this a contribution that would need an ICLA and I've not seen one I think we should evaluate where this will end up - in derby.jar or somewhere else?
        Hide
        Luigi Lauro added a comment - - edited

        Known issues and missing tidbits (show stoppers marked with *)

        0) NO TESTING (except very basic ones) DONE. Handle with care and give me feedback, so I can fix *******

        1) StorageFile.renameTo() not implemented yet (will do soon, it's easy enough). Hope derby can work without rename, in a fresh environment ********

        2) Check if PersistenceService implementation is meant to be thread-safe or not: currently my code do not synchronize in the StorageFile.createNewFile(), but this will need to be done, if PersistenceService does not give thread-safety guarantees. Probably, other methods will need to be synchronized as well. **********

        3) RandomAccess not implemented: to do it, sooner or later.

        4) Temporary files uses persistent entities: convert these to in-memory entities, to avoid using useful JNLP PersistenceService space and spamming the persistent entities list with temp files. Currently they are implemented using JNLP Persistent entities like normal files.

        5) I18N: to be done, will be last thing, when everything else is working.

        Show
        Luigi Lauro added a comment - - edited Known issues and missing tidbits (show stoppers marked with *) 0) NO TESTING (except very basic ones) DONE. Handle with care and give me feedback, so I can fix ******* 1) StorageFile.renameTo() not implemented yet (will do soon, it's easy enough). Hope derby can work without rename, in a fresh environment ******** 2) Check if PersistenceService implementation is meant to be thread-safe or not: currently my code do not synchronize in the StorageFile.createNewFile(), but this will need to be done, if PersistenceService does not give thread-safety guarantees. Probably, other methods will need to be synchronized as well. ********** 3) RandomAccess not implemented: to do it, sooner or later. 4) Temporary files uses persistent entities: convert these to in-memory entities, to avoid using useful JNLP PersistenceService space and spamming the persistent entities list with temp files. Currently they are implemented using JNLP Persistent entities like normal files. 5) I18N: to be done, will be last thing, when everything else is working.
        Hide
        Luigi Lauro added a comment - - edited

        JNLP Application Example (@David): Yes, I should have explained better how to make use of my work in derby.

        Let's split it in 2 steps:

        A) Enable my StorageFactory with derby: this is where you guys have to help me. I really don't know where to start here. I gave it a quick check, but from what I've seen, it seems the available factories are somewhat hard-coded into derby, and so some tinkering will be needed to get a derby starting using my JNLPStorageFactory.

        Any help here from someone who knows how to do this, is really appreciated.

        B) Setting up a Java Web Start Environment from where to run derby, in order to test the factory: unluckily, there is NO way to manually load the JNLP APIs implementation classes in a non java web start environment.

        I first tried to do it myself but got stuck in initializing problems when trying to load the PersistenceService implementation classes manually (you can read the whole story here: http://tech.groups.yahoo.com/group/junit/message/19132 ). Then queried both the official sun JNLP forums, and junit users forum, but to no avail.

        Therefore the only way I know to test my JNLPStorageFactory is to test it inside a real java web start environment.

        This means deploying a minimal derby jnlp application for testing. This is very easy to do (you only have to 'describe' your application with a xml-like .jnlp file, and then open this jnlp file with the java web start plugin), and you can check full info/docs/examples here: http://java.sun.com/products/javawebstart/

        The JNLP itself can be deployed next to the application jar, since the jar href supports relative paths, so no need to deploy jar&jnlp to a real web server. The only requisite is to bundle derby along with my classes in a proper jar, and write down a minimal jnlp file to start it from java web start.

        An example minimal JNLP:

        <?xml version="1.0" encoding="UTF-8"?>
        <jnlp codebase="http://db.apache.org/derby"/>
        <information>
        <title>Derby</title>
        </information>
        <resources>
        <jar href="derby.jar"/>
        </resources>
        <application-desc main-class="org.apache.derby.impl.MAIN_DERBY_CLASS"/>
        </jnlp>

        Show
        Luigi Lauro added a comment - - edited JNLP Application Example (@David): Yes, I should have explained better how to make use of my work in derby. Let's split it in 2 steps: A) Enable my StorageFactory with derby: this is where you guys have to help me. I really don't know where to start here. I gave it a quick check, but from what I've seen, it seems the available factories are somewhat hard-coded into derby, and so some tinkering will be needed to get a derby starting using my JNLPStorageFactory. Any help here from someone who knows how to do this, is really appreciated. B) Setting up a Java Web Start Environment from where to run derby, in order to test the factory: unluckily, there is NO way to manually load the JNLP APIs implementation classes in a non java web start environment. I first tried to do it myself but got stuck in initializing problems when trying to load the PersistenceService implementation classes manually (you can read the whole story here: http://tech.groups.yahoo.com/group/junit/message/19132 ). Then queried both the official sun JNLP forums, and junit users forum, but to no avail. Therefore the only way I know to test my JNLPStorageFactory is to test it inside a real java web start environment. This means deploying a minimal derby jnlp application for testing. This is very easy to do (you only have to 'describe' your application with a xml-like .jnlp file, and then open this jnlp file with the java web start plugin), and you can check full info/docs/examples here: http://java.sun.com/products/javawebstart/ The JNLP itself can be deployed next to the application jar, since the jar href supports relative paths, so no need to deploy jar&jnlp to a real web server. The only requisite is to bundle derby along with my classes in a proper jar, and write down a minimal jnlp file to start it from java web start. An example minimal JNLP: <?xml version="1.0" encoding="UTF-8"?> <jnlp codebase="http://db.apache.org/derby"/> <information> <title>Derby</title> </information> <resources> <jar href="derby.jar"/> </resources> <application-desc main-class="org.apache.derby.impl.MAIN_DERBY_CLASS"/> </jnlp>
        Hide
        Luigi Lauro added a comment -

        i18n (@David): Easy enough yes. Usually, when I do such 'spot' code, I don't bother with internationalization to focus on the real code. Once the code is working, I substitute the hardcoded string, with the proper resource lookup. Hell, it's even all automated with eclipse

        So, I'll check this in once things are done, it's a 20 minutes work, no more than that

        Show
        Luigi Lauro added a comment - i18n (@David): Easy enough yes. Usually, when I do such 'spot' code, I don't bother with internationalization to focus on the real code. Once the code is working, I substitute the hardcoded string, with the proper resource lookup. Hell, it's even all automated with eclipse So, I'll check this in once things are done, it's a 20 minutes work, no more than that
        Hide
        Luigi Lauro added a comment - - edited

        Code and JVM Version: Sorry guys here, but my project is currently targeting JVM1.6, and therefore I used some 1.6 stuff without even noticing (yes, isEmpty is 1.6, right).

        I think I'll go and rewrite this to be 1.5 compliant: going 1.4 could be done, but I think 1.5 is enough wide-spread now to go for it, especially since generics make code so much cleaner IMHO. Moreover, I coded against PersistenceService API, which if I remember correctly was changed a bit in 1.5, so I think 1.5 should be the 'sweet spot'.

        If you think I'm wrong and want me to adopt 1.4, I could do it, but I prefer going 1.5 directly if possible

        For 1.5 quick fix: just substitute the String.isEmpty with String.length >0, and split the new IOException("Msg", cause), in two lines, initializing cause separately. 3 lines fix.

        Show
        Luigi Lauro added a comment - - edited Code and JVM Version: Sorry guys here, but my project is currently targeting JVM1.6, and therefore I used some 1.6 stuff without even noticing (yes, isEmpty is 1.6, right). I think I'll go and rewrite this to be 1.5 compliant: going 1.4 could be done, but I think 1.5 is enough wide-spread now to go for it, especially since generics make code so much cleaner IMHO. Moreover, I coded against PersistenceService API, which if I remember correctly was changed a bit in 1.5, so I think 1.5 should be the 'sweet spot'. If you think I'm wrong and want me to adopt 1.4, I could do it, but I prefer going 1.5 directly if possible For 1.5 quick fix: just substitute the String.isEmpty with String.length >0, and split the new IOException("Msg", cause), in two lines, initializing cause separately. 3 lines fix.
        Hide
        Luigi Lauro added a comment -

        Full path in SVN DIFF (@ David): I will fix it for subsequent releases, this is probably eclipse that used the full path while produced the SVN DIFF (I know, I'm lazy ).

        Show
        Luigi Lauro added a comment - Full path in SVN DIFF (@ David): I will fix it for subsequent releases, this is probably eclipse that used the full path while produced the SVN DIFF (I know, I'm lazy ).
        Hide
        Luigi Lauro added a comment -

        Ok, I answer to you all on the several issues, in several upcoming comments

        Summary of the patch (@Dan): currently derby supports only several storage system for the database file persistence. The aim of this patch is to add a new Storage system for derby, built upon the JNLP PersistenceService API, which is a sun provided persistence service meant to be used in java web start environments (web apps or web-deployed desktop apps, as the one I'm developing), and is meant to be sandbox friendly (no 'give me full access to your HD' popups as per using the standard file factory).

        To get in depth, my implementation is currently made up of two implementation classes and an abstract class.

        AbstractStorageFactory: abstract base class implementing StorageFactory interface. This is meant to be a base class for StorageFactory implementation classes, and it's a sort of 'cleaned up' and rewritten from scratch version of derby BaseStorageFactory class. It provides basic initialization logic for needed database/temp directories, and it also gives a standard temporary files implementation, using standard files.

        JNLPStorageFactory: Java Web Start based StorageFactory implementation using JNLPStorageFile classes for file entities. All the persistence operations are delegated to JNLPStorageFile instances, which make use of the JNLP PersistenceService API provided by the Java Web Start framework, which is a sandbox friendly persistent storage available to JWS-enabled applications.

        Persistence is based on a cookie-like mechanism which stores persistent entities associated with URLs. We build hierarchy upon this flat structure by using a implicit naming hierarchy.

        Mapping is currently 1-1 (PersistenceEntity <---> Derby StorageFile) because I wanted to keep the implementation simple and straightforward, and this limits current applications of this factory till the sun PersistenceService implementation bug of maximum 255 persistence entries have been fixed (probably in next update, from the informations I have from Sun). This means the whole factory will fail horribly as soon as Derby asks for more than 255 storage entities.

        The current implementation use JNLP Storage entities even for temporary files (to avoid using standard files, which will defy the whole purpose of this factory - making a sandbox friendly JWS enabled factory), but I plan to plug in a in-memory implementation for the temp files, in order to avoid spamming the persistence storage with temporary files (especially since the 255 max entities bug, presently).

        JNLP Storage Entities have no hierarchy, so I had to code an implicit one into the name, using a special char (comma "," presently, as I thought it was the one least probable to be seen in URLs, but still valid).

        A JNLP Storage entity therefore is associate with a given URL, which is usually something like "http://APP_CODEBASE/DIR1,DIR2,DIR3,FILE" (ex: "http://www.derby.com/jnlp/directory,subdirectory,file")

        Currently, the implementation scan the whole persistent entities list to find the 'files' contained in a given directory, and parses all paths to find the entities that matches the needed path. This could be optimized by using data structures for indexing the entities, in order to avoid a full scan each time, but performance wise this should not be too much of a problem given the max 255 entities limit and the fact the PersistenceService is pretty efficient in retrieving the entities names. First make it work, then make it fast

        Show
        Luigi Lauro added a comment - Ok, I answer to you all on the several issues, in several upcoming comments Summary of the patch (@Dan): currently derby supports only several storage system for the database file persistence. The aim of this patch is to add a new Storage system for derby, built upon the JNLP PersistenceService API, which is a sun provided persistence service meant to be used in java web start environments (web apps or web-deployed desktop apps, as the one I'm developing), and is meant to be sandbox friendly (no 'give me full access to your HD' popups as per using the standard file factory). To get in depth, my implementation is currently made up of two implementation classes and an abstract class. AbstractStorageFactory: abstract base class implementing StorageFactory interface. This is meant to be a base class for StorageFactory implementation classes, and it's a sort of 'cleaned up' and rewritten from scratch version of derby BaseStorageFactory class. It provides basic initialization logic for needed database/temp directories, and it also gives a standard temporary files implementation, using standard files. JNLPStorageFactory: Java Web Start based StorageFactory implementation using JNLPStorageFile classes for file entities. All the persistence operations are delegated to JNLPStorageFile instances, which make use of the JNLP PersistenceService API provided by the Java Web Start framework, which is a sandbox friendly persistent storage available to JWS-enabled applications. Persistence is based on a cookie-like mechanism which stores persistent entities associated with URLs. We build hierarchy upon this flat structure by using a implicit naming hierarchy. Mapping is currently 1-1 (PersistenceEntity <---> Derby StorageFile) because I wanted to keep the implementation simple and straightforward, and this limits current applications of this factory till the sun PersistenceService implementation bug of maximum 255 persistence entries have been fixed (probably in next update, from the informations I have from Sun). This means the whole factory will fail horribly as soon as Derby asks for more than 255 storage entities. The current implementation use JNLP Storage entities even for temporary files (to avoid using standard files, which will defy the whole purpose of this factory - making a sandbox friendly JWS enabled factory), but I plan to plug in a in-memory implementation for the temp files, in order to avoid spamming the persistence storage with temporary files (especially since the 255 max entities bug, presently). JNLP Storage Entities have no hierarchy, so I had to code an implicit one into the name, using a special char (comma "," presently, as I thought it was the one least probable to be seen in URLs, but still valid). A JNLP Storage entity therefore is associate with a given URL, which is usually something like "http://APP_CODEBASE/DIR1,DIR2,DIR3,FILE" (ex: "http://www.derby.com/jnlp/directory,subdirectory,file") Currently, the implementation scan the whole persistent entities list to find the 'files' contained in a given directory, and parses all paths to find the entities that matches the needed path. This could be optimized by using data structures for indexing the entities, in order to avoid a full scan each time, but performance wise this should not be too much of a problem given the max 255 entities limit and the fact the PersistenceService is pretty efficient in retrieving the entities names. First make it work, then make it fast
        Hide
        David Van Couvering added a comment -

        Hi, Luigi. It looks like you built this with JDK 6 and used new Java 6 methods such as String.isEmpty() and IOException(String message, Throwable e).

        This would mean that this functionality would only work in Java SE 6 VMs. I think that's too restrictive, and I'm trying to adjust the code so it will compile under a 1.5 JDK.

        Also, all of your message strings are hardcoded. These need to be internationalized. I don't believe this is a showstopper (the network client code was checked in with hardcoded strings – I should know, I did the internationalization work), but it needs to be taken care of soon.

        I'll push up a patch that includes my changes once I get this puppy to compile and I seem to have something working.

        Thanks,

        David

        Show
        David Van Couvering added a comment - Hi, Luigi. It looks like you built this with JDK 6 and used new Java 6 methods such as String.isEmpty() and IOException(String message, Throwable e). This would mean that this functionality would only work in Java SE 6 VMs. I think that's too restrictive, and I'm trying to adjust the code so it will compile under a 1.5 JDK. Also, all of your message strings are hardcoded. These need to be internationalized. I don't believe this is a showstopper (the network client code was checked in with hardcoded strings – I should know, I did the internationalization work), but it needs to be taken care of soon. I'll push up a patch that includes my changes once I get this puppy to compile and I seem to have something working. Thanks, David
        Hide
        David Van Couvering added a comment -

        I made this change to indicate a desire to get this into 10.3 if possible. I am thinking this should be able to go in as long as it doesn't break the build, because it will never be used unless someone explicitly chooses to use it.

        Show
        David Van Couvering added a comment - I made this change to indicate a desire to get this into 10.3 if possible. I am thinking this should be able to go in as long as it doesn't break the build, because it will never be used unless someone explicitly chooses to use it.
        Hide
        David Van Couvering added a comment -

        Hi, Luigi. I was able to apply the patch. A couple of intiial comments. "Showstopper" means we have to get this resolved prior to checkin. The rest are desired but IMHO optional.

        Showstopper: - The JNLP jar file is not available with JDK 1.4, and also your code assumes JDK 1.5 (generics, annotations, for-each loops). Does this means you want this feature to be available only if the compiler (and thus the JRE) is 1.5 or higher? If so, we need to explicitly prevent this feature from being loaded at runtime if the user is running with a 1.4 VM. I'm not personally sure exactly how you do this, but others on the dev list have experience with this and you can ask about it.

        If you want it to be available with a 1.4 VM, then we need to fix the code to use only 1.4-compliant syntax, and we need to update BUILDING.txt to provide instructions for how to get jnlp.jar from http://java.sun.com/products/javawebstart/download-jnlp.html.

        • I have to concur with Dan, it would really help to provide instructions (say in the javadoc for the JNLP storage factory) for how you enable this and how you use it. I'm not a JNLP expert and I'm kind of scratching my head around how to get started...
        • You might think about checking in a demo Java Web Start application to the java/demo directory, with instructions, to make it easier for people to figure out how to use this feature.

        I'll probably have more feedback after playing around with this for a bit.

        David

        Show
        David Van Couvering added a comment - Hi, Luigi. I was able to apply the patch. A couple of intiial comments. "Showstopper" means we have to get this resolved prior to checkin. The rest are desired but IMHO optional. Showstopper: - The JNLP jar file is not available with JDK 1.4, and also your code assumes JDK 1.5 (generics, annotations, for-each loops). Does this means you want this feature to be available only if the compiler (and thus the JRE) is 1.5 or higher? If so, we need to explicitly prevent this feature from being loaded at runtime if the user is running with a 1.4 VM. I'm not personally sure exactly how you do this, but others on the dev list have experience with this and you can ask about it. If you want it to be available with a 1.4 VM, then we need to fix the code to use only 1.4-compliant syntax, and we need to update BUILDING.txt to provide instructions for how to get jnlp.jar from http://java.sun.com/products/javawebstart/download-jnlp.html . I have to concur with Dan, it would really help to provide instructions (say in the javadoc for the JNLP storage factory) for how you enable this and how you use it. I'm not a JNLP expert and I'm kind of scratching my head around how to get started... You might think about checking in a demo Java Web Start application to the java/demo directory, with instructions, to make it easier for people to figure out how to use this feature. I'll probably have more feedback after playing around with this for a bit. David
        Hide
        David Van Couvering added a comment -

        Hi, Luigi, I don't know how you generated this patch, but it has the full path to your sandbox hardcoded in the patch file. I'm trying to fix that, but did you do an 'svn diff' from the root of the source tree, or did you do something else?

        Show
        David Van Couvering added a comment - Hi, Luigi, I don't know how you generated this patch, but it has the full path to your sandbox hardcoded in the patch file. I'm trying to fix that, but did you do an 'svn diff' from the root of the source tree, or did you do something else?
        Hide
        Daniel John Debrunner added a comment -

        Is there a summary in the patch of how one would use this functionality? I couldn't see one.

        Show
        Daniel John Debrunner added a comment - Is there a summary in the patch of how one would use this functionality? I couldn't see one.
        Hide
        David Van Couvering added a comment -

        I think this is great you have done all this work, Luigi. I am going to take a look. I believe that this feature has enough value that we should really try to see if we can get this in, even if it's not "perfect."

        I am not a store expert but I can see if I can figure out how to run some tests on this. If others with more experience in this area have ideas, please pass them on, or maybe you can even run the tests.

        P.S. I voted for that issue...

        Show
        David Van Couvering added a comment - I think this is great you have done all this work, Luigi. I am going to take a look. I believe that this feature has enough value that we should really try to see if we can get this in, even if it's not "perfect." I am not a store expert but I can see if I can figure out how to run some tests on this. If others with more experience in this area have ideas, please pass them on, or maybe you can even run the tests. P.S. I voted for that issue...
        Hide
        Luigi Lauro added a comment -

        Cleaned the implementation a bit and improved java docs.

        Now the implementation, feature-wise, should be 95% complete.

        What I need now is someone with a higher knowledge of the derby internals than me (or more available time to spend to tinker with it), to pick up my implementation for code review and to give it a try with the storage test suite and with some real world tests, to see what I did wrong and have to be fixed.

        I can help with any problem, but I seriously need some help for the testing/reviewing, otherwise I would have to spend too much time finding out how to properly test my implementation with Derby, and this is not something I can afford to do presently (other higher priorities).

        Anyone willing to lend me a hand? I would really appreciate it, and be very willing to listen and fix any things that may comes up, and we would soon have a new cool storage implementation for this wonderful database

        Also, I'm still trying to find a way to make derby use the smallest number of files and disk space possible, in order to make it work with the given tight constraints of the current BUGGED (Check: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6539564 and VOTE!) Sun implementation of PersistenceService API.

        David, Jeff, Dan, Andrew, Francois... I'm talking with you guys!

        Thanks in advance for any help.

        Show
        Luigi Lauro added a comment - Cleaned the implementation a bit and improved java docs. Now the implementation, feature-wise, should be 95% complete. What I need now is someone with a higher knowledge of the derby internals than me (or more available time to spend to tinker with it), to pick up my implementation for code review and to give it a try with the storage test suite and with some real world tests, to see what I did wrong and have to be fixed. I can help with any problem, but I seriously need some help for the testing/reviewing, otherwise I would have to spend too much time finding out how to properly test my implementation with Derby, and this is not something I can afford to do presently (other higher priorities). Anyone willing to lend me a hand? I would really appreciate it, and be very willing to listen and fix any things that may comes up, and we would soon have a new cool storage implementation for this wonderful database Also, I'm still trying to find a way to make derby use the smallest number of files and disk space possible, in order to make it work with the given tight constraints of the current BUGGED (Check: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6539564 and VOTE!) Sun implementation of PersistenceService API. David, Jeff, Dan, Andrew, Francois... I'm talking with you guys! Thanks in advance for any help.
        Hide
        Luigi Lauro added a comment -

        Reworked the implementation in a cleaner one.

        Decided to go after a simple abstract base class (AbstractStorageFactory) similar to derby BaseStorageFactory, but reworked it into a cleaner better version of it.

        JNLPStorageFactory build upon this base class and delegates to JNLPStorageFile for actual storage operations.

        Still a lot of work to do, bear with me

        Show
        Luigi Lauro added a comment - Reworked the implementation in a cleaner one. Decided to go after a simple abstract base class (AbstractStorageFactory) similar to derby BaseStorageFactory, but reworked it into a cleaner better version of it. JNLPStorageFactory build upon this base class and delegates to JNLPStorageFile for actual storage operations. Still a lot of work to do, bear with me
        Hide
        Luigi Lauro added a comment -

        Cleaned up a bit the implementation, fixed some bugs, changed names into more suitable ones, added more javadoc comments for SimpleStorageFactory.

        JNLPStorageFactory is still very raw and untested/uncommented.

        I'm currently focusing on 'finishing' my alternative BaseStorageFactory (SimpleStorageFactory) as an abstract base class with a simpler approach for easier implementations (I will provide a memory and JNLP one, and then the other Dir/CP/Jar could be converted into using this new base class).

        Feedback on anything VERY VERY welcome.

        Show
        Luigi Lauro added a comment - Cleaned up a bit the implementation, fixed some bugs, changed names into more suitable ones, added more javadoc comments for SimpleStorageFactory. JNLPStorageFactory is still very raw and untested/uncommented. I'm currently focusing on 'finishing' my alternative BaseStorageFactory (SimpleStorageFactory) as an abstract base class with a simpler approach for easier implementations (I will provide a memory and JNLP one, and then the other Dir/CP/Jar could be converted into using this new base class). Feedback on anything VERY VERY welcome.
        Hide
        Luigi Lauro added a comment -

        Attaching my latest versions.

        I'm coding around a VirtualStorageFactory abstract class, and a VirtualStorageFile. VirtualStorageFile implements basic logic and delegates to VirtualStorageFactory for the actual storage manipulation.

        VirtualStorageFactory base class is a reworked from scratch 'BaseStorageFactory' which instead of forcing implementors to give both StorageFile and StorageFactory implementation, allows them only to implement a few 'CRUD' create/delete/rename/delete methods, and build upon these to give a working StorageFactory/StorageFile implementation.

        Ideally, if things go as I plan, every storage implementation could be re-written around this new base class if needed, or this could be used as a new base for developing new implementations in a easier way.

        Have a look at the abstract class (VirtualStorageFactory) and the delegating implentation of StorageFile (VirtualStorageFile).

        I also include a starting JNLP implementation of VirtualStorageFactory.

        Javadoc comments comes mostly from interfaces (copy & paste), and VERY FEW TESTS DONE YET, this is very alpha code, if you wanna help go through it and suggest fixes/improvements/whatever to get things rolling for real.

        P.S. Classes names are ugly, I plan to change those soon.

        Show
        Luigi Lauro added a comment - Attaching my latest versions. I'm coding around a VirtualStorageFactory abstract class, and a VirtualStorageFile. VirtualStorageFile implements basic logic and delegates to VirtualStorageFactory for the actual storage manipulation. VirtualStorageFactory base class is a reworked from scratch 'BaseStorageFactory' which instead of forcing implementors to give both StorageFile and StorageFactory implementation, allows them only to implement a few 'CRUD' create/delete/rename/delete methods, and build upon these to give a working StorageFactory/StorageFile implementation. Ideally, if things go as I plan, every storage implementation could be re-written around this new base class if needed, or this could be used as a new base for developing new implementations in a easier way. Have a look at the abstract class (VirtualStorageFactory) and the delegating implentation of StorageFile (VirtualStorageFile). I also include a starting JNLP implementation of VirtualStorageFactory. Javadoc comments comes mostly from interfaces (copy & paste), and VERY FEW TESTS DONE YET, this is very alpha code, if you wanna help go through it and suggest fixes/improvements/whatever to get things rolling for real. P.S. Classes names are ugly, I plan to change those soon.
        Hide
        Luigi Lauro added a comment -

        Done some discoveries about JNLP PersistenceService limits, which are harsher than I initially thought.

        I'm investigating if a derby storage can be easily done given these limits.

        You can read about discoveries and discussion here:

        http://thread.gmane.org/gmane.comp.apache.db.derby.devel/39038
        http://thread.gmane.org/gmane.comp.apache.db.derby.devel/39044

        Show
        Luigi Lauro added a comment - Done some discoveries about JNLP PersistenceService limits, which are harsher than I initially thought. I'm investigating if a derby storage can be easily done given these limits. You can read about discoveries and discussion here: http://thread.gmane.org/gmane.comp.apache.db.derby.devel/39038 http://thread.gmane.org/gmane.comp.apache.db.derby.devel/39044
        Hide
        Luigi Lauro added a comment -

        Started JNLPStorageFactory, almost nothing done presently, I'm still in the brainstorming/planning phase, trying to decide the best way to proceeed.

        Discussion about approach and ideas found here: http://thread.gmane.org/gmane.comp.apache.db.derby.devel/38843

        Feel free to contribute

        Show
        Luigi Lauro added a comment - Started JNLPStorageFactory, almost nothing done presently, I'm still in the brainstorming/planning phase, trying to decide the best way to proceeed. Discussion about approach and ideas found here: http://thread.gmane.org/gmane.comp.apache.db.derby.devel/38843 Feel free to contribute
        Show
        Luigi Lauro added a comment - Some pointers: Thread regarding this issue: http://thread.gmane.org/gmane.comp.apache.db.derby.user/6365 JNLP APIs: http://java.sun.com/javase/6/docs/jre/api/javaws/jnlp/ PersistenceService API: http://java.sun.com/javase/6/docs/jre/api/javaws/jnlp/javax/jnlp/PersistenceService.html
        Hide
        Daniel John Debrunner added a comment -
        Show
        Daniel John Debrunner added a comment - For those that don't know the API I found this tutorial: http://webdia.cem.itesm.mx:8005/web/java/tutorial/deployment/webstart/api.html which also has links to the PersistentService API http://java.sun.com/javase/6/docs/jre/api/javaws/jnlp/javax/jnlp/PersistenceService.html

          People

          • Assignee:
            Unassigned
            Reporter:
            Luigi Lauro
          • Votes:
            3 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:

              Development