Company model tables are not fully cleaned up

• Key: JDO-375
• Type: Bug
• Status: Resolved
• Resolution: Fixed
• Priority: Major
• Assignee: Erik Bengtson
• Reporter: Erik Bengtson
• Component/s: tck2
• Affects Version/s: JDO 2 rc1
• Fix Version/s: JDO 2 final

Description

Tables from company model does not appear to be clean after run.

I can't say if this is a JPOX or TCK issue, because I did not dig into the issue.

On second run, exceptions are like

> 1)
> test(org.apache.jdo.tck.api.persistencemanager.extent.GetExtentWithInstancesMadePersistentViaReachability)javax.jdo.JDODataStoreException:
> Insert request failed: INSERT INTO applicationidentity0.COMPANIES
> ("NAME",FOUNDEDDATE,ADDRID,CITY,COUNTRY,"STATE",STREET,ZIPCODE,ID,DISCRIMINATOR)
> VALUES (?,?,?,?,?,?,?,?,?,?)
> at org.jpox.store.rdbms.request.InsertRequest.execute(InsertRequest.java:393)
> at org.jpox.store.rdbms.table.ClassTable.insert(ClassTable.java:2171)
> at org.jpox.store.StoreManager.insert(StoreManager.java:757)
> at
> org.jpox.state.StateManagerImpl.internalMakePersistent(StateManagerImpl.java:3456)
> at org.jpox.state.StateManagerImpl.makePersistent(StateManagerImpl.java:3429)
> at
> org.jpox.AbstractPersistenceManager.internalMakePersistent(AbstractPersistenceManager.java:1150)
> at
> org.jpox.AbstractPersistenceManager.makePersistent(AbstractPersistenceManager.java:1205)
> at
> org.apache.jdo.tck.api.persistencemanager.extent.GetExtentWithInstancesMadePersistentViaReachability.createObjects(GetExtentWithInstancesMadePersistentViaReachability.java:72)
> at
> org.apache.jdo.tck.api.persistencemanager.extent.GetExtentWithInstancesMadePersistentViaReachability.test(GetExtentWithInstancesMadePersistentViaReachability.java:63)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at org.apache.jdo.tck.JDO_Test.runBare(JDO_Test.java:237)
> at org.apache.jdo.tck.util.BatchTestRunner.doRun(BatchTestRunner.java:107)
> at org.apache.jdo.tck.util.BatchTestRunner.start(BatchTestRunner.java:147)
> at org.apache.jdo.tck.util.BatchTestRunner.main(BatchTestRunner.java:122)
> NestedThrowablesStackTrace:
> ERROR 23505: The statement was aborted because it would have caused a duplicate
> key value in a unique or primary key constraint or unique index identified by
> 'COMPS_PK' defined on 'COMPANIES'.
> at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
> at org.apache.derby.impl.sql.execute.IndexChanger.insertAndCheckDups(Unknown
> Source)
> at org.apache.derby.impl.sql.execute.IndexChanger.doInsert(Unknown Source)
> at org.apache.derby.impl.sql.execute.IndexChanger.insert(Unknown Source)
> at org.apache.derby.impl.sql.execute.IndexSetChanger.insert(Unknown Source)
> at org.apache.derby.impl.sql.execute.RowChangerImpl.insertRow(Unknown Source)
> at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown
> Source)
> at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source)
> at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
> at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
> at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown
> Source)
> at org.apache.derby.impl.jdbc.EmbedPreparedStatement.execute(Unknown Source)
> at
> com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.execute(NewProxyPreparedStatement.java:911)
> at
> org.jpox.store.rdbms.ParamLoggingPreparedStatement.execute(ParamLoggingPreparedStatement.java:212)
> at org.jpox.store.rdbms.request.Request.executeUpdate(Request.java:88)
> at org.jpox.store.rdbms.request.InsertRequest.execute(InsertRequest.java:267)
> at org.jpox.store.rdbms.table.ClassTable.insert(ClassTable.java:2171)
> at org.jpox.store.StoreManager.insert(StoreManager.java:757)
> at
> org.jpox.state.StateManagerImpl.internalMakePersistent(StateManagerImpl.java:3456)
> at org.jpox.state.StateManagerImpl.makePersistent(StateManagerImpl.java:3429)
> at
> org.jpox.AbstractPersistenceManager.internalMakePersistent(AbstractPersistenceManager.java:1150)
> at
> org.jpox.AbstractPersistenceManager.makePersistent(AbstractPersistenceManager.java:1205)
> at
> org.apache.jdo.tck.api.persistencemanager.extent.GetExtentWithInstancesMadePersistentViaReachability.createObjects(GetExtentWithInstancesMadePersistentViaReachability.java:72)
> at
> org.apache.jdo.tck.api.persistencemanager.extent.GetExtentWithInstancesMadePersistentViaReachability.test(GetExtentWithInstancesMadePersistentViaReachability.java:63)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at org.apache.jdo.tck.JDO_Test.runBare(JDO_Test.java:237)
> at org.apache.jdo.tck.util.BatchTestRunner.doRun(BatchTestRunner.java:107)
> at org.apache.jdo.tck.util.BatchTestRunner.start(BatchTestRunner.java:147)
> at org.apache.jdo.tck.util.BatchTestRunner.main(BatchTestRunner.java:122)
> 2)
> test(org.apache.jdo.tck.api.persistencemanager.extent.GetExtentWithSubclasses)javax.jdo.JDODataStoreException:
> Insert request failed: INSERT INTO applicationidentity0.PERSONS
> (ADDRID,CITY,COUNTRY,"STATE",STREET,ZIPCODE,FIRSTNAME,BIRTHDATE,MIDDLENAME,LASTNAME,PERSONID,DISCRIMINATOR)
> VALUES (?,?,?,?,?,?,?,?,?,?,?,?)
> at org.jpox.store.rdbms.request.InsertRequest.execute(InsertRequest.java:393)
> at org.jpox.store.rdbms.table.ClassTable.insert(ClassTable.java:2171)
> at org.jpox.store.StoreManager.insert(StoreManager.java:757)
> at
> org.jpox.state.StateManagerImpl.internalMakePersistent(StateManagerImpl.java:3456)
> at org.jpox.state.StateManagerImpl.makePersistent(StateManagerImpl.java:3429)
> at
> org.jpox.AbstractPersistenceManager.internalMakePersistent(AbstractPersistenceManager.java:1150)
> at
> org.jpox.AbstractPersistenceManager.makePersistent(AbstractPersistenceManager.java:1205)
> at
> org.apache.jdo.tck.api.persistencemanager.extent.GetExtentWithSubclasses.createObjects(GetExtentWithSubclasses.java:86)
> at
> org.apache.jdo.tck.api.persistencemanager.extent.GetExtentWithSubclasses.test(GetExtentWithSubclasses.java:73)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at org.apache.jdo.tck.JDO_Test.runBare(JDO_Test.java:237)
> at org.apache.jdo.tck.util.BatchTestRunner.doRun(BatchTestRunner.java:107)
> at org.apache.jdo.tck.util.BatchTestRunner.start(BatchTestRunner.java:147)
> at org.apache.jdo.tck.util.BatchTestRunner.main(BatchTestRunner.java:122)
> NestedThrowablesStackTrace:
> ERROR 23505: The statement was aborted because it would have caused a duplicate
> key value in a unique or primary key constraint or unique index identified by
> 'EMPS_PK' defined on 'PERSONS'.
> at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
> at org.apache.derby.impl.sql.execute.IndexChanger.insertAndCheckDups(Unknown
> Source)
> at org.apache.derby.impl.sql.execute.IndexChanger.doInsert(Unknown Source)
> at org.apache.derby.impl.sql.execute.IndexChanger.insert(Unknown Source)
> at org.apache.derby.impl.sql.execute.IndexSetChanger.insert(Unknown Source)
> at org.apache.derby.impl.sql.execute.RowChangerImpl.insertRow(Unknown Source)
> at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown
> Source)
> at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source)
> at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
> at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
> at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown
> Source)
> at org.apache.derby.impl.jdbc.EmbedPreparedStatement.execute(Unknown Source)
> at
> com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.execute(NewProxyPreparedStatement.java:911)
> at
> org.jpox.store.rdbms.ParamLoggingPreparedStatement.execute(ParamLoggingPreparedStatement.java:212)
> at org.jpox.store.rdbms.request.Request.executeUpdate(Request.java:88)
> at org.jpox.store.rdbms.request.InsertRequest.execute(InsertRequest.java:267)
> at org.jpox.store.rdbms.table.ClassTable.insert(ClassTable.java:2171)
> at org.jpox.store.StoreManager.insert(StoreManager.java:757)
> at
> org.jpox.state.StateManagerImpl.internalMakePersistent(StateManagerImpl.java:3456)
> at org.jpox.state.StateManagerImpl.makePersistent(StateManagerImpl.java:3429)
> at
> org.jpox.AbstractPersistenceManager.internalMakePersistent(AbstractPersistenceManager.java:1150)
> at
> org.jpox.AbstractPersistenceManager.makePersistent(AbstractPersistenceManager.java:1205)
> at
> org.apache.jdo.tck.api.persistencemanager.extent.GetExtentWithSubclasses.createObjects(GetExtentWithSubclasses.java:86)
> at
> org.apache.jdo.tck.api.persistencemanager.extent.GetExtentWithSubclasses.test(GetExtentWithSubclasses.java:73)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at org.apache.jdo.tck.JDO_Test.runBare(JDO_Test.java:237)
> at org.apache.jdo.tck.util.BatchTestRunner.doRun(BatchTestRunner.java:107)
> at org.apache.jdo.tck.util.BatchTestRunner.start(BatchTestRunner.java:147)
> at org.apache.jdo.tck.util.BatchTestRunner.main(BatchTestRunner.java:122)
> 3)
> test(org.apache.jdo.tck.api.persistencemanager.lifecycle.MakePersistentAndInstancesNotReachable)javax.jdo.JDODataStoreException:
> Insert request failed: INSERT INTO applicationidentity0.COMPANIES
> ("NAME",FOUNDEDDATE,ADDRID,CITY,COUNTRY,"STATE",STREET,ZIPCODE,ID,DISCRIMINATOR)
> VALUES (?,?,?,?,?,?,?,?,?,?)
> at org.jpox.store.rdbms.request.InsertRequest.execute(InsertRequest.java:393)
> at org.jpox.store.rdbms.table.ClassTable.insert(ClassTable.java:2171)
> at org.jpox.store.StoreManager.insert(StoreManager.java:757)
> at
> org.jpox.state.StateManagerImpl.internalMakePersistent(StateManagerImpl.java:3456)
> at org.jpox.state.StateManagerImpl.makePersistent(StateManagerImpl.java:3429)
> at
> org.jpox.AbstractPersistenceManager.internalMakePersistent(AbstractPersistenceManager.java:1150)
> at
> org.jpox.AbstractPersistenceManager.makePersistent(AbstractPersistenceManager.java:1205)
> at
> org.apache.jdo.tck.api.persistencemanager.lifecycle.MakePersistentAndInstancesNotReachable.createObjects(MakePersistentAndInstancesNotReachable.java:85)
> at
> org.apache.jdo.tck.api.persistencemanager.lifecycle.MakePersistentAndInstancesNotReachable.test(MakePersistentAndInstancesNotReachable.java:69)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at org.apache.jdo.tck.JDO_Test.runBare(JDO_Test.java:237)
> at org.apache.jdo.tck.util.BatchTestRunner.doRun(BatchTestRunner.java:107)
> at org.apache.jdo.tck.util.BatchTestRunner.start(BatchTestRunner.java:147)
> at org.apache.jdo.tck.util.BatchTestRunner.main(BatchTestRunner.java:122)
> NestedThrowablesStackTrace:
> ERROR 23505: The statement was aborted because it would have caused a duplicate
> key value in a unique or primary key constraint or unique index identified by
> 'COMPS_PK' defined on 'COMPANIES'.
> at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
> at org.apache.derby.impl.sql.execute.IndexChanger.insertAndCheckDups(Unknown
> Source)
> at org.apache.derby.impl.sql.execute.IndexChanger.doInsert(Unknown Source)
> at org.apache.derby.impl.sql.execute.IndexChanger.insert(Unknown Source)
> at org.apache.derby.impl.sql.execute.IndexSetChanger.insert(Unknown Source)
> at org.apache.derby.impl.sql.execute.RowChangerImpl.insertRow(Unknown Source)
> at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown
> Source)
> at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source)
> at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
> at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
> at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown
> Source)
> at org.apache.derby.impl.jdbc.EmbedPreparedStatement.execute(Unknown Source)
> at
> com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.execute(NewProxyPreparedStatement.java:911)
> at
> org.jpox.store.rdbms.ParamLoggingPreparedStatement.execute(ParamLoggingPreparedStatement.java:212)
> at org.jpox.store.rdbms.request.Request.executeUpdate(Request.java:88)
> at org.jpox.store.rdbms.request.InsertRequest.execute(InsertRequest.java:267)
> at org.jpox.store.rdbms.table.ClassTable.insert(ClassTable.java:2171)
> at org.jpox.store.StoreManager.insert(StoreManager.java:757)
> at
> org.jpox.state.StateManagerImpl.internalMakePersistent(StateManagerImpl.java:3456)
> at org.jpox.state.StateManagerImpl.makePersistent(StateManagerImpl.java:3429)
> at
> org.jpox.AbstractPersistenceManager.internalMakePersistent(AbstractPersistenceManager.java:1150)
> at
> org.jpox.AbstractPersistenceManager.makePersistent(AbstractPersistenceManager.java:1205)
> at
> org.apache.jdo.tck.api.persistencemanager.lifecycle.MakePersistentAndInstancesNotReachable.createObjects(MakePersistentAndInstancesNotReachable.java:85)
> at
> org.apache.jdo.tck.api.persistencemanager.lifecycle.MakePersistentAndInstancesNotReachable.test(MakePersistentAndInstancesNotReachable.java:69)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at org.apache.jdo.tck.JDO_Test.runBare(JDO_Test.java:237)
> at org.apache.jdo.tck.util.BatchTestRunner.doRun(BatchTestRunner.java:107)
> at org.apache.jdo.tck.util.BatchTestRunner.start(BatchTestRunner.java:147)
> at org.apache.jdo.tck.util.BatchTestRunner.main(BatchTestRunner.java:122)
> FAILURES!!!
> Error summary:
> 001 error: SQL Exception: The statement was aborted because it would have
> caused a duplicate key value in a unique or primary key constraint or unique
> index identified by 'EMPS_PK' defined on 'PERSONS'.
> 002 errors: SQL Exception: The statement was aborted because it would have
> caused a duplicate key value in a unique or primary key constraint or unique
> index identified by 'COMPS_PK' defined on 'COMPANIES'.
> derby-app-pm-junit.txt:
> ** Tests run: 099, Time: 032 seconds. Failures: 0, Errors: 3
> Excluded tests: [org.apache.jdo.tck.enhancement.FieldAccessModified,
> org.apache.jdo.tck.enhancement.ImplementsPersistenceCapable]
>

Comments

[Michael Bouschen]

I found one issue with the cleanup code in the CompletenessTest: it retriveves the list of tearDownClasses from the reader that is used to create the transient graph for comparison. But this reader returns the PC classes and not the PC interfaces. The attached patch JDO-375.patch fixes this bug.

However, cleanup still does not work. The cleanup code uses the following pattern (as Criag mentioned in the other email):
  Query query = new Query();
  query.setClass(ICompany.class);
  query.setCandidates(pm.getExtent(ICompany.class, false));
  pm.deletePersistentAll((Collection)query.execute());
I added some debug messages and figured out that the query using the extent of a persistent interface does not return any instances.

[Craig Russell]

The patch looks good. I agree it needs to be fixed for final.

[Craig Russell]

Please include the patch for CompanyModelPMClass to change IMedicalInsurance.class to MedicalInsurance.class in this fix.

[Michael Bouschen]

I checked in the patch fixing getting the tearDown classes from the correct reader plus fixing list of tearDown classes in CompanyFactoryPMClass into the trunk and the 2.0 branch (see revision 398354).

I think the remaining issue from preventing a correct cleanup after the test is, that the extent of a persistent interface does not include any instances. I'm assigning this issue back to Erik.

[Erik Bengtson]

JPOX now return instances to extent. interfaces test now fails during tearDown while deleting instances due to a constraint failure.

It would be nice to have meaningfull constraints names...

    [java] 001 error: SQL Exception: DELETE on table 'PERSONS' caused a violati
on of foreign key constraint 'SQL060425095724181' for key (12). The statement h
as been rolled back.

[Craig Russell]

There are several constraints that have their own names, but unfortunately not all constraints are named.

But it looks like the only unnamed constraints on persons are the self-referencing manager, mentor, hradvisor, and insurance. Insurance is removed first so it looks like it's a self-constraint that is causing the problem.

This isn't an issue for persistent classes. Do you have the same strategy for deleting instances of generated classes as you do for user classes?

[Michael Bouschen]

The attached patch JDO-375-2.patch implements Craig's proposal to remove classes Person and Emplyoee from the list of teardown classes in CompletenessTest. I propose this change should go into the trunk and the 2.0 branch.

The patch might change the order in which instances get deleted and this might affect the cleanup in the companyPMInterface test configuration. Please let me know.

[Craig Russell]

The patch looks good for the trunk and branch/2.0.

[Michael Bouschen]

Checked in patch JDO-375-2.patch into the trunk and 2.0 branch (see revision 398434).

[Craig Russell]

The attached patch adds names to the foreign key constraints for the company model in schema 0.

[Michael Bouschen]

The fknames patch looks good!

[Andy Jefferson]

Attached are jars built from JPOX CVS HEAD. I can confirm the fix works and have run the complete TCK twice through with success. Good work!

[Michael Bouschen]

I checked in Craig's fknames patch (see revison 398624).

With the latest JPOX version the cleanup issue is resolved.

[Michael Bouschen]

Added fknames patch to the 2.0 branch (see revision 398629).