Convert upgrade tests to Junit

• Key: DERBY-2217
• Type: Improvement
• Status: Closed
• Resolution: Fixed
• Priority: Minor
• Assignee: Daniel John Debrunner
• Reporter: Daniel John Debrunner
• Component/s: Test
• Fix Version/s: 10.3.1.4

Description

Also support running against multiple old versions.

Comments

[Daniel John Debrunner]

Revision 494201 (trunk) has a working JUnit based upgrade test that handles multiple old versions and does not hit sealing issues seen by the older harness based test.
The test uses the same property to specifiy the location of the old released jars *but* in a different way to the old test.

The old test used

${derbyTesting.jar.path}/M.m/derby.jar

The new setup uses (to allow multiple releases at the same major.minor

${derbyTesting.jar.path}/M.m.f.p/lib/derby.jar

The set of old releases tested against is given by a list in _Suite. This was to allow a releases folder to contain a number of previous releases and have derbyTesting.jar.path point to it.
e.g.

/derby/releases/10.0.2.1
/derby/releases/10.1.3.1
etc.

I have not changed the tools/testing folder to the new scheme, this new JUnit upgrade test does not yet include all the cases from the old test. If we wanted to switch to the new scheme incrementally then I could change the property used by the JUnit test to something else (e.g. derbyTesting.oldReleases) and then we could run the tests in parallel until the new one is a super-set of the old.

The test can be run as a standard JUnit test., though it has to be run using the _Suite class:

org.apache.derbyTesting.functionTests.tests.upgradeTests._Suite

If the system property derbyTesting.jar.path is not set, then no tests will be run.

If for an old release the folder ${derbyTesting.jar.path}/M.m.f.p/lib does not exist then that old release will be skipped.

It's not hooked up into any standard run (e.g. suites.All) due to the changes need to jar layout and that in trunk upgrading from 10.0 fails (need to check out why).

'm also not sure if all the old releases should be in a code line, or just a sub-set. The test would handle either situation and maybe running all the old versions could be left to those that want to (by pointing the property to a folder with all the old releases).
(BTW currently with five old releases the test takes around 25 seconds)

[Myrna van Lunteren]

I generally like the new approach.

I have some small concerns though.
1. This test presumably relies on the new changes to support junit tests...And thus it is not applicable for 10.2 branch.
    Where does that leave us for the 10.2 branch? Is it acceptable to continue automated runs there with an upgrade from 10.1.1.0 to 10.2.(latest)?
   I tried to use derbyTesting.jar with the new tests, and 10.2.2.0 derby.jar to see if that was an approach that would work, but I get errors.

2. I noticed the automatic specification of the jarlocation (done in upgradeTests/build.xml via a _app file) has been dropped. I think that's ok - but it assumes we will have to run suites.All with the new property passed in, or the upgrade tests will get skipped.

[Daniel John Debrunner]

Thanks for looking at the code Myrna.

On 1) currently it's not my itch to solve the 10.2 tests. I too tried the 10.2 derby.jar with the new tests and had some issues, though I thought I'd got it to work partially, since I managed to test the upgrade from 10.0 was not broken in 10.2. (see DERBY-2221). I'll re-run again and see what happens.

On 2) I was hoping that I could set the property for the old release directory automatically when running the tests via the ant targets in build.xml.

[Daniel John Debrunner]

I changed the junit upgrade test to use a different property from the old test to describe the location of the old jars.
The new test uses:

derbyTesting.oldReleasePath

[Daniel John Debrunner]

Need metadata tests in JUnit form for the upgrade tests.

[Daniel John Debrunner]

Current state.
The test currently passes with these JVMs (running trunk against all old releases except the incubator 10.0 release)

Sun JDK 1.4.2
Sun JDK 1.5
IBM SDK 1.5

With Java SE 6 the test fails, seems to find the classes for the new version when the old version is expected and successfully picked up by other JVMs.
Most likely the test configuration is looking for the JDBC 4 data sources which don't exist in the old versions of Derby, thus they resolve to the current version. Probably need to force JDBC 3 data sources as the data source class to use if the old version is older than 10.2.2.0.

With these two J2ME/CDC/Foundation JVMs the test hangs.

IBM WCTME 5.7
IBM WEME 6.1

[Daniel John Debrunner]

Test converted to JUnit, hooked into suites.All if property pointing to old releases is set.

[Rick Hillegas]

This is great work. Thanks, Dan. I find that the JUnit-ized upgrade test runs cleanly against all of the releases from 10.1.1.0 onward. However, I see 4 errors when I run against 10.0.2.1:

1) testNegValueSupportedLogRecord(org.apache.derbyTesting.functionTests.tests.upgradeTests.Changes10_3)java.sql.SQLException: 'SYSCS_UTIL.SYSCS_INPLACE_COMPRESS_TABLE' is not recognized as a function or procedure.

...

2) testGetTablesModify(org.apache.derbyTesting.functionTests.tests.jdbcapi.DatabaseMetaDataTest)java.sql.SQLException: Use of 'XML' requires database to be upgraded from version 10.0 to version 10.1 or later.

...

3) testNegValueSupportedLogRecord(org.apache.derbyTesting.functionTests.tests.upgradeTests.Changes10_3)ERROR 42Y03: 'SYSCS_UTIL.SYSCS_INPLACE_COMPRESS_TABLE' is not recognized as a function or procedure.

...

4) Upgrade Phase: UPGRADEjava.sql.SQLException: Failed to start database 'singleUse/oneuse0', see the next exception for details.

[Daniel John Debrunner]

4) is probably the same as DERBY-2221. I stopped running on 10.0 when I hit that issue.