Issue Details (XML | Word | Printable)

Key: DERBY-3988
Type: Improvement Improvement
Status: Closed Closed
Resolution: Fixed
Priority: Major Major
Assignee: Rick Hillegas
Reporter: Rick Hillegas
Votes: 0
Watchers: 0
Operations

If you were logged in you would be able to see more operations.
Derby

Set the Java 6 complilation path based on the Java 5 compilation path so that the JDBC4 support is always built

Created: 12/Dec/08 07:18 PM   Updated: 04/May/09 06:22 PM
Component/s: Build tools
Affects Version/s: 10.5.1.1
Fix Version/s: 10.5.1.1

Time Tracking:
Not Specified

File Attachments:
  Size
File Licensed for inclusion in ASF works derby-3988-01-aa-alwaysCompileJDBC4.diff 2008-12-12 08:13 PM Rick Hillegas 5 kB
File Licensed for inclusion in ASF works derby-3988-01-ab-alwaysCompileJDBC4.diff 2008-12-15 04:17 PM Rick Hillegas 7 kB
File Licensed for inclusion in ASF works derby-3988-01-ac-alwaysCompileJDBC4.diff 2008-12-16 02:46 PM Rick Hillegas 7 kB
File Licensed for inclusion in ASF works derby-3988-01-ad-alwaysCompileJDBC4.diff 2008-12-19 08:39 PM Rick Hillegas 10 kB
File Licensed for inclusion in ASF works derby-3988-02-aa-fixJarProperties.diff 2009-01-12 02:56 PM Rick Hillegas 1 kB
File Licensed for inclusion in ASF works derby-3988-02-ab-fixJarProperties.diff 2009-01-12 06:51 PM Rick Hillegas 4 kB
Issue Links:
Dependants
 
Incorporates
 
dependent
 

Resolution Date: 08/Jan/09 04:48 PM


 Description  « Hide
Now that we have checked in the JDBC4 stubs, it ought to be possible to build the JDBC4 support with just the Java 5 libraries and compiler. Set java16compile.classpath=${jdbc4stubs}:${java15compile.classpath}.

 All   Comments   Work Log   Change History   Subversion Commits      Sort Order: Ascending order - Click to sort in descending order
Rick Hillegas added a comment - 12/Dec/08 08:13 PM
Attaching derby-3988-01-aa-alwaysCompileJDBC4.diff. This patch makes it possible to build the entire Derby product given only the Derby source and a Java 5 environment. Makes the following changes:

1) Sets java16compile.classpath=${jdbc4stubs}:${java15compile.classpath}

2) Eliminates the jdk16 conditional variable.

3) Changes the Java 6 targets to compile to Java 5 byte code and to use the default compiler.


Touches the following files:

M build.xml

Changes needed for (1).


M java/engine/org/apache/derby/impl/jdbc/build.xml
M java/engine/org/apache/derby/iapi/jdbc/build.xml
M java/engine/org/apache/derby/jdbc/build.xml
M java/engine/build.xml
M java/testing/org/apache/derbyTesting/functionTests/tests/jdbc4/build.xml
M java/client/build.xml

Changes needed for (2) and (3).

Knut Anders Hatlen added a comment - 13/Dec/08 03:44 PM
I think removing the jdk16 variable and using the same compiler for all the classes is a good move, as it makes it easier to understand the build and which compiler has been used for which classes. But as I said in DERBY-3966, I'm not sure if compiling all classes with JDK 5 is what we want. The issues I see are:

  1) We're restricted to using a subset of Java SE 6 in code that is only ever used on Java SE 6

  2) We must compile the classes with target=1.5 instead of target=1.6, so we don't get the benefits of the new class format, like faster class loading

Is there any reason why we couldn't just take the full step and require JDK 6 instead of JDK 5? It's been two years since Java SE 6 was released, so I don't think it's unreasonable to expect that any serious Java development environment contains a Java 6 compiler these days.

Some possible improvements to the patch:

  - the compile_jdbc4 targets don't need fork="yes" anymore, I think

  - instead of always setting target=1.5 we could have a property that had the value 1.5 or 1.6 depending of the actual compiler used

  - instead of making jdbc4_modules_edit in java/engine/build.xml non-optional, we should remove that target and make the necessary changes to modules.properties

Knut Anders Hatlen added a comment - 15/Dec/08 10:11 AM
Bryan Pendleton <bpendleton@amberpoint.com> writes:

>> Is there any reason why we couldn't just take the full step and
>> require JDK 6 instead of JDK 5?
>
> To be clear, you propose to require JDK 6 for *building* Derby, but
> the code, once built, would continue to run in the current set of
> environments (JDK 1.4+, small environment, etc.)?
>
> If so, that approach seems fine to me.

Yes, that's what I meant. Require JDK 6 (or later) to build Derby, but
Derby should continue to run on the same platforms as before.

Rick Hillegas added a comment - 15/Dec/08 02:45 PM
Thanks for the feedback, Knut and Bryan. For the record, there is still no production version of the Java 6 JDK available for 32-bit Intel Macs. There is a beta version of the Java 6 JDK for that platform, but it has problems. I don't know when a production release will appear. I think that this is an important platform and I would be reluctant to require a Java 6 compiler at this time.

Rick Hillegas added a comment - 15/Dec/08 04:17 PM
Attaching derby-3988-01-ab-alwaysCompileJDBC4.diff. This patch makes the 3 improvements suggested by Knut:

1) Doesn't fork while compiling the JDBC4 support

2) Compiles the JDBC4 support to Java 6 byte code if possible

3) Removes the "optional" prefixes on the JDBC4 modules in modules.properties

Touches the following files:

M java/engine/org/apache/derby/impl/jdbc/build.xml
M java/engine/org/apache/derby/iapi/jdbc/build.xml
M java/engine/org/apache/derby/jdbc/build.xml
M java/engine/org/apache/derby/modules.properties
M java/engine/build.xml
M java/testing/org/apache/derbyTesting/functionTests/tests/jdbc4/build.xml
M java/client/build.xml
M build.xml

Knut Anders Hatlen added a comment - 16/Dec/08 10:41 AM
The patch looks good to me. Perhaps it's better to have 16 in the name of the new property (java16compiler.level or something) in case we want to use a similar mechanism for other versions in the future (new JDKs, new JDBC revisions...). I guess this is an OK compromise for now. We can revisit how to add Java 6 methods if/when we want to introduce some of them in the code.

Just one more question: what's the recommended setting for java16compile.classpath for those of us who have JDK 6 installed? For all the other *compile.classpath properties we recommend that developers point them to the real thing (that is, a real implementation of that version of the class library) to get maximum compile-time checking. For java16compile.classpath, pointing to the real thing would give less compile-time checking, as it doesn't necessarily discover that a change will break the build for those who don't have JDK 6.

Rick Hillegas added a comment - 16/Dec/08 02:46 PM
Attaching derby-3988-01-ac-alwaysCompileJDBC4.diff. This renames compilerLevel -> compilerLevel16 as Knut suggested.

Hi Knut,

This patch doesn't change what java16compile.classpath should be set to. I have been setting java16compile.classpath to the list of jar files in my Java 6 JDK, just as java15compile.classpath is automatically set to the list of jar files in my Java 5 JDK.

Please get me back on track if I'm not following you: I think you're wondering about what happens if someone, who compiles using Java 6, checks some logic into the JDBC4 driver which mentions a new class introduced by Java 6. Today (before applying this patch) that would not be noticed by other developers who compile using Java 5. After this patch is applied, you are right, this will break the build for people who compile using Java 5. Probably we would have to back out that patch until we decided how to handle these Java 6 references.

The only solutions which have been proposed so far are:

1) Require the Java 6 compiler rather than the Java 5 compiler

2) Check in some more Java 6 stubs

I agree that we can probably wait to solve this problem until we have a concrete situation in front of us. By then, Java 6 may be supported on all the platforms we care about and (1) may be an uncontroversial solution.

Are you planning to add references to new Java 6 classes/methods soon? If so, do you have some other proposals in mind besides (1) and (2)?

Knut Anders Hatlen added a comment - 16/Dec/08 03:33 PM
> Are you planning to add references to new Java 6 classes/methods soon?

No.

> If so, do you have some other proposals in mind besides (1) and (2)?

You also have this option:

3) Build JDBC 4.0 only when using JDK 6 or later (though automatically, as you suggested, without setting the jdk16 property)

Then JDBC 4.0 support will be built automatically for everyone who has installed JDK 6, it won't break the build for those who are still using JDK 5, and it doesn't prevent using the full power of JDK 6 in code that is written for JDK 6. The downside is that the JDBC 4.0 driver is not compiled on platforms where JDK 6 is not yet available.

But I'm fine with the latest proposal for now.

Rick Hillegas added a comment - 16/Dec/08 04:36 PM
Right. I should have been more precise. I'm most interested in solutions which satisfy these goals:

A) Give us a build that works out-of-the-box, building the entire Derby product

B) Let us take advantage of improvements introduced by advanced VMs

Sounds like you're ok with the current patch. I intend to commit it soon unless someone raises an objection.

Rick Hillegas added a comment - 16/Dec/08 06:25 PM
Committed derby-3988-01-ac-alwaysCompileJDBC4.diff at subversion revision 727106.

Myrna van Lunteren added a comment - 16/Dec/08 09:18 PM - edited
I didn't get a chance to test out the patch before your commit.

Now, when I build with sun jvms/jres all is fine, but when I build with ibm jres, I get this:
compile_jdbc4:
    [javac] Compiling 5 source files to C:\derbyt\svn\trunk\classes
    [javac] C:<myrnaspathto>trunk\java\engine\org\apache\derby\iapi\jdbc\BrokeredCallableStatement40.java:26: cannot access java.sql.Blob
    [javac] bad class file: c:<myrnaspathtoibm16>\jre\lib\sql.jar(java/sql/Blob.class)
    [javac] class file has wrong version 50.0, should be 49.0
    [javac] Please remove or make sure it appears in the correct subdirectory of the classpath.
    [javac] import java.sql.Blob;
    [javac] ^
    [javac] 1 error
    [javac] Compile failed; see the compiler error output for details.

I see errors like this for:
- \java\engine\org\apache\derby\iapi\jdbc\BrokeredCallableStatement40.java:26: cannot access java.sql.Blob
- java\engine\org\apache\derby\impl\jdbc\EmbedCallableStatement40.java:26: cannot access java.sql.Blob
- java\engine\org\apache\derby\jdbc\Driver40.java:24: cannot access java.sql.DatabaseMetaData
- java\testing\org\apache\derbyTesting\functionTests\tests\jdbc4\BlobClobTestSetup.java:79: cannot access java.sql.SQLException
- java\client\org\apache\derby\client\ClientPooledConnection40.java:24: cannot access java.sql.PreparedStatement
- java\testing\org\apache\derbyTesting\functionTests\tests\jdbc4\BlobClobTestSetup.java:79: cannot access java.sql.SQLException

My ant.properties looks like this:
-----------------------------
j14lib=c:<myrnaspathtoibm142>/jre/lib
j15lib=c:<myrnaspathtoibm15>/jre/lib
jdk16=c:<myrnaspathtoibm16>
j16lib=${jdk16}/jre/lib

java14compile.classpath=${j14lib}/core.jar;${j14lib}/server.jar;${j14lib}/xml.jar;${j14lib}/ibmjcefw.jar;${j14lib}/security.jar
java15compile.classpath=${j15lib}/vm.jar;${j15lib}/core.jar;${j15lib}/server.jar;${j15lib}/xml.jar;${j15lib}/ibmjcefw.jar;${j15lib}/security.jar
java16compile.classpath=${j16lib}/sql.jar;${j16lib}/xml.jar;${j16lib}/ibmjcefw.jar;${j16lib}/security.jar;${j16lib}/rt.jar;${j16lib}/security.jar;${j16lib}/vm.jar;${j16lib}/java.util.jar;${j16lib}/jndi.jar

jsr169compile.classpath=C:<myrnaspathtoweme6.1>/lib/jclFoundation11/classes.zip;C:<myrnaspathtoweme6.1>/lib/jdbc.jar

junit=c:<myrnaspathtojunit>/java/junit.jar

build.compiler=modern
proceed=true
sane=true
#sane=false
----------------------------------

Where the 'myrnaspathto...' is the appropriate paths.

Suggestions?

Rick Hillegas added a comment - 16/Dec/08 09:40 PM
Hi Myrna,

Thanks for running that experiment. Could you try the following:

1) Put the following line into your ant.properties

printCompilerProperties=true

2) Then do a clean build with your Sun environment

3) Then do a clean build with your IBM environment

Early on in your builds, you will see a block of diagnostics reporting what the build thinks your classpaths and compiler levels are. Could you let us know what these blocks are for your Sun and IBM builds? The blocks look like this:

     [echo] Before setting properties: compilerLevel16 = 1.5
     [echo] Before setting properties: jsr169compile.classpath = ${jsr169compile.classpath}
     [echo] Before setting properties: j14lib = ${j14lib}
     [echo] Before setting properties: java14compile.classpath = ${java14compile.classpath}
     [echo] Before setting properties: j15lib = ${j15lib}
     [echo] Before setting properties: java15compile.classpath = ${java15compile.classpath}
     [echo] Before setting properties: java16compile.classpath = ${java16compile.classpath}
     [echo] After setting properties: compilerLevel16 = 1.5
     [echo] After setting properties: jsr169compile.classpath = /Users/rh161140/derby/mainline/trunk/classes/stubs/jsr169:/System/Library/Frameworks/JavaVM.framework/Versions/1.4/Classes/charsets.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.4/Classes/classes.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.4/Classes/dt.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.4/Classes/jce.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.4/Classes/jsse.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.4/Classes/laf.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.4/Classes/sunrsasign.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.4/Classes/ui.jar
     [echo] After setting properties: j14lib = ${j14lib}
     [echo] After setting properties: java14compile.classpath = /System/Library/Frameworks/JavaVM.framework/Versions/1.4/Classes/charsets.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.4/Classes/classes.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.4/Classes/dt.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.4/Classes/jce.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.4/Classes/jsse.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.4/Classes/laf.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.4/Classes/sunrsasign.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.4/Classes/ui.jar
     [echo] After setting properties: j15lib = ${j15lib}
     [echo] After setting properties: java15compile.classpath = /System/Library/Frameworks/JavaVM.framework/Versions/1.5/Classes/charsets.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.5/Classes/classes.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.5/Classes/dt.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.5/Classes/jce.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.5/Classes/jconsole.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.5/Classes/jsse.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.5/Classes/laf.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.5/Classes/ui.jar
     [echo] After setting properties: java16compile.classpath = /Users/rh161140/derby/mainline/trunk/classes/stubs/jdbc4:/System/Library/Frameworks/JavaVM.framework/Versions/1.5/Classes/charsets.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.5/Classes/classes.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.5/Classes/dt.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.5/Classes/jce.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.5/Classes/jconsole.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.5/Classes/jsse.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.5/Classes/laf.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.5/Classes/ui.jar

Thanks!

Myrna van Lunteren added a comment - 16/Dec/08 11:38 PM
That's interesting - my ibm jvm environment is missing the jdbc4 stubs after the second setting of compilerLevel 1.5.
As an experiment, I added the hardcode path to my classes/stubs/jdbc4 to my java16compile.classpath and it took care of most of the errors - only the java\testing\org\apache\derbyTesting\functionTests\tests\jdbc4\BlobTest.java:146: cannot access java.util.HashMap warning was still there.
Without that addition, this is what was printed out with my env set to use sun jvms (with some continuation markers [\] to improve the view in jira)
--------------------------------------------
printCompilerProperties:
     [echo] Before setting properties: compilerLevel16 = 1.5
     [echo] Before setting properties: jsr169compile.classpath = C:/jvms/weme6.1/ive/lib/jclFoundation11/classes.zip;C:/jvms/weme6.1/ive/lib/jdbc.jar
     [echo] Before setting properties: j14lib = c:/jvms/jdk142/jre/lib
     [echo] Before setting properties: java14compile.classpath = ${java14compile.classpath}
     [echo] Before setting properties: j15lib = c:/jvms/jdk15/jre/lib
     [echo] Before setting properties: java15compile.classpath = ${java15compile.classpath}
     [echo] Before setting properties: java16compile.classpath = ${java16compile.classpath}
[setJdkProperties] Setting property java14compile.classpath to c:\jvms\jdk142\jre\lib\charsets.jar;c:\jvms\jdk142\jre\lib\jce.jar;[\]
c:\jvms\jdk142\jre\lib\jsse.jar;c:\jvms\jdk142\jre\lib\plugin.jar;c:\jvms\jdk142\jre\lib\plugin_g.jar;[\]
c:\jvms\jdk142\jre\lib\rt.jar;c:\jvms\jdk142\jre\lib\sunrsasign.jar
[setJdkProperties] Setting property java15compile.classpath to c:\jvms\jdk15\jre\lib\charsets.jar;c:\jvms\jdk15\jre\lib\deploy.jar;c:\jvms\jdk15\jre\lib\javaws.jar;c:\jvms\jdk15\jre\lib\jce.jar;c:\jvms\jdk15\jre\lib\jsse.jar;c:\jvms\jdk15\jre\lib\plugin.jar;c:\jvms\jdk15\jre\lib\rt.jar
[setJdkProperties] Setting property java14compile.classpath to c:\jvms\jdk142\jre\lib\charsets.jar;c:\jvms\jdk142\jre\lib\jce.jar;c:\jvms\jdk142\jre\lib\jsse.jar;c:\jvms\jdk142\jre\lib\plugin.jar;c:\jvms\jdk142\jre\lib\plugin_g.jar;c:\jvms\jdk142\jre\lib\rt.jar;c:\jvms\jdk142\jre\lib\sunrsasign.jar
[setJdkProperties] Setting property java15compile.classpath to c:\jvms\jdk15\jre\lib\charsets.jar;c:\jvms\jdk15\jre\lib\deploy.jar;c:\jvms\jdk15\jre\lib\javaws.jar;c:\jvms\jdk15\jre\lib\jce.jar;c:\jvms\jdk15\jre\lib\jsse.jar;c:\jvms\jdk15\jre\lib\plugin.jar;c:\jvms\jdk15\jre\lib\rt.jar

printCompilerProperties:
     [echo] After setting properties: compilerLevel16 = 1.5
     [echo] After setting properties: jsr169compile.classpath = C:/jvms/weme6.1/ive/lib/jclFoundation11/classes.zip;C:/jvms/weme6.1/ive/lib/jdbc.jar
     [echo] After setting properties: j14lib = c:/jvms/jdk142/jre/lib
     [echo] After setting properties: java14compile.classpath = c:\jvms\jdk142\jre\lib\charsets.jar;c:\jvms\jdk142\jre\lib\jce.jar;c:\jvms\jdk142\jre\lib\jsse.jar;[\]
c:\jvms\jdk142\jre\lib\plugin.jar;c:\jvms\jdk142\jre\lib\plugin_g.jar;c:\jvms\jdk142\jre\lib\rt.jar;c:\jvms\jdk142\jre\lib\sunrsasign.jar
     [echo] After setting properties: j15lib = c:/jvms/jdk15/jre/lib
     [echo] After setting properties: java15compile.classpath = c:\jvms\jdk15\jre\lib\charsets.jar;c:\jvms\jdk15\jre\lib\deploy.jar;[\]
c:\jvms\jdk15\jre\lib\javaws.jar;c:\jvms\jdk15\jre\lib\jce.jar;c:\jvms\jdk15\jre\lib\jsse.jar;c:\jvms\jdk15\jre\lib\plugin.jar;c:\jvms\jdk15\jre\lib\rt.jar
     [echo] After setting properties: java16compile.classpath = C:\derbyt\svn\trunk/classes/stubs/jdbc4:c:\jvms\jdk15\jre\lib\charsets.jar;[\]
c:\jvms\jdk15\jre\lib\deploy.jar;c:\jvms\jdk15\jre\lib\javaws.jar;c:\jvms\jdk15\jre\lib\jce.jar;c:\jvms\jdk15\jre\lib\jsse.jar;[\]
c:\jvms\jdk15\jre\lib\plugin.jar;c:\jvms\jdk15\jre\lib\rt.jar
--------------------------------------------
And this is what was printed out with my env set to use ibm jvms:
--------------------------------------------
printCompilerProperties:
     [echo] Before setting properties: compilerLevel16 = 1.5
     [echo] Before setting properties: jsr169compile.classpath = C:/jvms/weme6.1/lib/jclFoundation11/classes.zip;C:/jvms/weme6.1/lib/jdbc.jar
     [echo] Before setting properties: j14lib = c:/jvms/ibm142/jre/lib
     [echo] Before setting properties: java14compile.classpath = c:/jvms/ibm142/jre/lib/core.jar;c:/jvms/ibm142/jre/lib/server.jar;[\]
c:/jvms/ibm142/jre/lib/xml.jar;c:/jvms/ibm142/jre/lib/ibmjcefw.jar;c:/jvms/ibm142/jre/lib/security.jar
     [echo] Before setting properties: j15lib = c:/jvms/ibm15/jre/lib
     [echo] Before setting properties: java15compile.classpath = c:/jvms/ibm15/jre/lib/vm.jar;c:/jvms/ibm15/jre/lib/core.jar;[\]
c:/jvms/ibm15/jre/lib/server.jar;c:/jvms/ibm15/jre/lib/xml.jar;c:/jvms/ibm15/jre/lib/ibmjcefw.jar;c:/jvms/ibm15/jre/lib/security.jar
     [echo] Before setting properties: java16compile.classpath = c:/jvms/ibm16/jre/lib/sql.jar;c:/jvms/ibm16/jre/lib/xml.jar;[\]
c:/jvms/ibm16/jre/lib/ibmjcefw.jar;c:/jvms/ibm16/jre/lib/security.jar;c:/jvms/ibm16/jre/lib/rt.jar;c:/jvms/ibm16/jre/lib/security.jar;[\]
c:/jvms/ibm16/jre/lib/vm.jar;c:/jvms/ibm16/jre/lib/java.util.jar;c:/jvms/ibm16/jre/lib/jndi.jar

printCompilerProperties:
     [echo] After setting properties: compilerLevel16 = 1.5
     [echo] After setting properties: jsr169compile.classpath = C:/jvms/weme6.1/lib/jclFoundation11/classes.zip;C:/jvms/weme6.1/lib/jdbc.jar
     [echo] After setting properties: j14lib = c:/jvms/ibm142/jre/lib
     [echo] After setting properties: java14compile.classpath = c:/jvms/ibm142/jre/lib/core.jar;c:/jvms/ibm142/jre/lib/server.jar;[\]
c:/jvms/ibm142/jre/lib/xml.jar;c:/jvms/ibm142/jre/lib/ibmjcefw.jar;c:/jvms/ibm142/jre/lib/security.jar
     [echo] After setting properties: j15lib = c:/jvms/ibm15/jre/lib
     [echo] After setting properties: java15compile.classpath = c:/jvms/ibm15/jre/lib/vm.jar;c:/jvms/ibm15/jre/lib/core.jar;[\]
c:/jvms/ibm15/jre/lib/server.jar;c:/jvms/ibm15/jre/lib/xml.jar;c:/jvms/ibm15/jre/lib/ibmjcefw.jar;c:/jvms/ibm15/jre/lib/security.jar
     [echo] After setting properties: java16compile.classpath = c:/jvms/ibm16/jre/lib/sql.jar;c:/jvms/ibm16/jre/lib/xml.jar;[\]
c:/jvms/ibm16/jre/lib/ibmjcefw.jar;c:/jvms/ibm16/jre/lib/security.jar;c:/jvms/ibm16/jre/lib/rt.jar;c:/jvms/ibm16/jre/lib/security.jar;[\]
c:/jvms/ibm16/jre/lib/vm.jar;c:/jvms/ibm16/jre/lib/java.util.jar;c:/jvms/ibm16/jre/lib/jndi.jar
----------------------------------------------------

Rick Hillegas added a comment - 17/Dec/08 12:21 AM
Thanks for the extra information, Myrna. It looks to me like:

BEFORE this patch:

1) In your Sun environment, the build did NOT compile the JDBC4 driver

2) In your IBM environment, the build compiled the JDBC4 driver using the IBM Java 6 compiler and the IBM Java 6 libraries

AFTER this patch:

1) In your Sun environment, the build compiled the JDBC4 driver using the Sun Java 5 compiler and Derby's stubs (which were compiled by the Sun Java 5 compiler)

2) In your IBM environment, the build tried to compile the JDBC4 driver using the IBM Java 5 compiler and the IBM Java 6 libraries. This is what raised the version mismatch error.

Here are some possible solutions to this problem:

A) If you are using the Java 5 environment, then the build should ignore the setting of java16compile.classpath and force it to be the classpath composed of the jdbc4 stubs followed by the java5compile.classpath

B) If you are using the Java 5 environment and java16compile.classpath is set, then the build should raise an error saying that it can't handle the mixed signals

I would like to solicit other opinions and sleep on this.

And you raised another issue: I seem to be going blind and I don't see the java.util.HashMap warning. Can you post that by itself?

Thanks for running these experiments!

Myrna van Lunteren added a comment - 17/Dec/08 12:55 AM
Re hashMap warning...You're not blind, I am...After adding the stubs/jdbc4 dir to the java16compile.classpath, I get a new set of warnings (and I only spotted 1):
- java\engine\org\apache\derby\iapi\jdbc\BrokeredConnection40.java:33: cannot access java.util.Properties
- java\engine\org\apache\derby\impl\jdbc\EmbedConnection40.java:33: cannot access java.util.HashMap
- java\engine\org\apache\derby\jdbc\Driver40.java:48: cannot access java.util.Properties
- java\client\org\apache\derby\client\am\LogicalConnection40.java:32: cannot access java.util.Properties
- java\testing\org\apache\derbyTesting\functionTests\tests\jdbc4\BlobTest.java:146: cannot access java.util.HashMap

And re 40 driver getting built, I can assure you it got built.
I can svn update to that state and build and send the result, but in the mean time, I've got a non-working environment.
Can you back out your latest commit until we've figured out a solution?

Rick Hillegas added a comment - 17/Dec/08 01:42 PM
Hi Myrna: I have backed out the commit at subversion revision 727379.

Myrna van Lunteren added a comment - 17/Dec/08 08:51 PM
Thanks Rick...

I noticed something interesting - on a (linux) machine where I had set up to build with the ibm jvms there was no problem after your changes. I think the relevant difference between the two systems is the ant version.
So, your changes worked (even with ibm jvms) with ant 1.6.5, but not with ant 1.7...


Myrna van Lunteren added a comment - 18/Dec/08 07:51 PM
Another experiment showed that with my tree sync-ed up to revision 727106, if I remove the java16compile.classpath from the $HOME/ant.properties, the build succeeds with ibm jvm also.
The stubs/jdbc4 directory is then prepended.

Re building Driver40 with my sun setup; I maybe should have mentioned that when I build with the sun jvms, there is no need for the java*compile.classpath settings...

As another experiment, I set up my sun jdk settings in $HOME/ant.properties, but added a java16compile.classpath setting, and all was still fine.
So, I suspect there is a bug in the ibm jvm where it picks the wrong libraries (1.6) if both 1.5 and 1.6 are available; maybe the difference I attributed to ant version difference then is a difference in the windows jvm.

Anyway, I have a workaround to 'cope' with this quirky behavior and the changes as per 727106; remove the java16compile.classpath setting.
The build then uses the stubs (which I assume must get built with 1.6?) and 1.5 for everything else...(?) I hope I understand this correctly.

It's a little frustrating that I'll have to use a different ant.properties for the different branches but I can live with that....


Rick Hillegas added a comment - 18/Dec/08 09:17 PM
Thanks for the additional information, Myrna. I have reproduced your original results on Ubuntu Linux. I ran the following experiments:

1) My environment has Sun and IBM JDKs for both Java 5 and Java 6

2) I built Derby with each of those four compilers. For each compiler, I ran the build twice: first with no setting for java16compile.classpath, then later with java16compile.classpath set to the Java 6 libraries supplied by the compiler's vendor. So a total of eight builds:

i) Sun Java 5 compiler, java16compile.classpath NOT initially set
ii) Sun Java 5 compiler, java16compile.classpath initially set to Sun Java 6 libraries
iii) Sun Java 6 compiler, java16compile.classpath NOT initially set
iv) Sun Java 6 compiler, java16compile.classpath initially set to Sun Java 6 libraries
v) IBM Java 5 compiler, java16compile.classpath NOT initially set
vi) IBM Java 5 compiler, java16compile.classpath initially set to IBM Java 6 libraries
vii) IBM Java 6 compiler, java16compile.classpath NOT initially set
viii) IBM Java 6 compiler, java16compile.classpath initially set to IBM Java 6 libraries

All of the builds succeeded except for (vi), That build died with a "class file has wrong version 50.0, should be 49.0" error when compiling java.sql.Blob, as you discovered.


Rick Hillegas added a comment - 18/Dec/08 09:27 PM
Hi Myrna,

I also want to follow up on another comment you made:

> Re building Driver40 with my sun setup; I maybe should have mentioned that when I build with the sun jvms, there is no need for the java*compile.classpath settings...

This says to me that the PropertySetter program is correctly finding your Sun JDKs. That's good. Has the build been failing for you when you use an IBM compiler but don't set the java*compile.classpath variables? That would be bad. That suggests to me that PropertySetter isn't smart enough to find your IBM JDKs. We may be able to fix PropertySetter so that it finds the IBM JDKs on your machine.

Thanks,
-Rick

Myrna van Lunteren added a comment - 19/Dec/08 12:26 AM
I believe that at one time, I found it necessary to set these properties or the PropertySetter didn't correctly find my ibm jdks. I seem to remember it assumed some default names for the installations.
However, I did some more experimenting just now, (still with my tree environment at the 727106 revision level) including having a sun jdk compiler and getting ibm jdks, and it all seems fine.
So, I don't know anymore what made me decide to need the *compile.classpath variables...


Rick Hillegas added a comment - 19/Dec/08 08:39 PM
Attaching derby-3988-01-ad-alwaysCompileJDBC4.diff. This is identical to the previous patch except that it adds a check to PropertySetter. The check aborts the build if the IBM Java 5 compiler is being used and java16compile.classpath is set. In that case, the build aborts, telling the user to either use a Java 6 compiler or unset java16compile.classpath.

Myrna, please let me know if this is an acceptable solution. I think that you should be able to use the same ant.properties for your 10.5 and 10.4 builds as long as you use a Java 6 compiler for both.

Thanks.

Myrna van Lunteren added a comment - 20/Dec/08 01:32 AM
I did some more experimenting, and I think this is acceptable.
Thank you for your patience.
+1 to commit patch *ad*

Rick Hillegas added a comment - 22/Dec/08 01:52 PM
Thanks for running more experiments and for the +1, Myrna. I have committed derby-3988-01-ad-alwaysCompileJDBC4.diff at subversion revision 728693.

Rick Hillegas added a comment - 12/Jan/09 02:56 PM
Attaching derby-3988-02-aa-fixJarProperties.diff. This uncomments the JDBC4 data sources in the list of extra classes which need to be pulled in when we build the jars. When the jdk16 property was deprecated, these classes were no longer pulled into derby.jar. Touches the following file:

M tools/jar/extraDBMSclasses.properties

Knut Anders Hatlen added a comment - 12/Jan/09 03:52 PM
The fixJarProperties patch looks good to me. By the way, the jdk16 property is still used in build.xml (for building javadoc and running JUnit tests) even though it's been removed from the build instructions.

Should the same change also be made to tools/jar/dnc.properties?

Could we now remove this target from build.xml?

  <target name="jdbc4_classlist_edit" if="jdk16">
    <!-- uncomment jdbc4 classes if jdk 1.6 is available -->
    <echo level="info" message="uncommenting jdbc4 classes"/>
    <replaceregexp file="${jar.classlist}"
     match="^#jdbc4_optional_(.*)"
     replace="\1"
     byline="true"/>
  </target>

Rick Hillegas added a comment - 12/Jan/09 06:51 PM
Thanks, Knut. Attaching derby-3988-02-ab-fixJarProperties.diff. This adjusts dnc.properties and the master build.xml as you suggest.

Knut Anders Hatlen added a comment - 13/Jan/09 09:03 AM
Thanks, Rick. The ab patch looks fine. +1 to commit.

Rick Hillegas added a comment - 13/Jan/09 02:52 PM
Thanks, Knut. Committed derby-3988-02-ab-fixJarProperties.diff at subversion revision 734146.

Dag H. Wanvik added a comment - 14/Jan/09 12:00 AM
I can confirm that after fixJarProperties was committed I can now run off jars again, thanks, Rick.