Test api.persistencemanager.OptimisticFailure hangs

• Key: JDO-72
• Type: Bug
• Status: Resolved
• Resolution: Fixed
• Priority: Major
• Assignee: Andy Jefferson
• Reporter: Craig Russell
• Component/s: tck2

Description

This test is designed to create conflicts in the database from two different JDO transactions. The changes in the cache must not be visible in the datastore or timeouts will occur. The exception here occurs when the second optimistic JDO transaction attempts to read a row that has been changed in the cache by the first optimistic JDO transaction.
    private void runTestOptimistic(PersistenceManager pm1,
                                   PersistenceManager pm2,
                                   PersistenceManager pm3) {
        if (!isOptimisticSupported()) {
           if (debug)
               logger.debug("OptimisticFailure tests not run; Optimistic not supported");
           return;
        }

        Transaction tx1 = pm1.currentTransaction();
        Transaction tx2 = pm2.currentTransaction();
        Transaction tx3 = pm3.currentTransaction();
        try {
           tx1.setOptimistic(true);
           tx2.setOptimistic(true);
           
           // create four instances to test
           tx1.begin();
           pm1.makePersistent(p1);
           pm1.makePersistent(p2);
           pm1.makePersistent(p3);
           pm1.makePersistent(p4);
           pm1.makePersistent(p5);
           p1oid = pm1.getObjectId(p1);
           p2oid = pm1.getObjectId(p2);
           p3oid = pm1.getObjectId(p3);
           p4oid = pm1.getObjectId(p4);
           p5oid = pm1.getObjectId(p5);
           tx1.commit();
           
           // update/delete the instances in tx1
           tx1.begin();
           PCPoint p1tx1 = (PCPoint)pm1.getObjectById(p1oid, true);
           PCPoint p2tx1 = (PCPoint)pm1.getObjectById(p2oid, true);
           PCPoint p3tx1 = (PCPoint)pm1.getObjectById(p3oid, true);
           PCPoint p4tx1 = (PCPoint)pm1.getObjectById(p4oid, true);
           p1tx1.setX(101);
           p2tx1.setX(201);
           pm1.deletePersistent(p3tx1);
           pm1.deletePersistent(p4tx1);
           
           // update/delete the instances in tx2
           tx2.begin();
*** PCPoint p1tx2 = (PCPoint)pm2.getObjectById(p1oid, true); *** this is where the test hangs ***
           PCPoint p2tx2 = (PCPoint)pm2.getObjectById(p2oid, true);
           PCPoint p3tx2 = (PCPoint)pm2.getObjectById(p3oid, true);
           PCPoint p4tx2 = (PCPoint)pm2.getObjectById(p4oid, true);
           PCPoint p5tx2 = (PCPoint)pm2.getObjectById(p5oid, true);
           p1tx2.setX(102);
// pm2.deletePersistent(p2tx2); // this should fail but succeeds due to an RI bug
           p3tx2.setX(202);
           pm2.deletePersistent(p4tx2);
           p5tx2.setX(502); // this change should not be committed
           Set expectedFailedObjects = new HashSet();
           expectedFailedObjects.add(p1tx2);
// expectedFailedObjects.add(p2tx2);
           expectedFailedObjects.add(p3tx2);
           expectedFailedObjects.add(p4tx2);
           
           // commit tx1 (should succeed)
           tx1.commit();
           tx1 = null;
           
           // commit tx2 (should fail)
           try {
               tx2.commit();
               fail(ASSERTION_FAILED, "concurrent commit not detected");
           }
           catch (JDOOptimisticVerificationException ex) {
               // verify the correct information in the exception


RUN OptimisticFailure.test[INFO] tck - Exception during setUp or runtest: <javax.jdo.JDODataStoreException: Fetch request failed: SELECT PCPOINT.X,PCPOINT.Y,PCPOINT.ID FROM PCPOINT WHERE (PCPOINT.ID = ?)
    [java] NestedThrowables:
    [java] SQL Exception: A lock could not be obtained within the time requested>javax.jdo.JDODataStoreException: Fetch request failed: SELECT PCPOINT.X,PCPOINT.Y,PCPOINT.ID FROM PCPOINT WHERE (PCPOINT.ID = ?)
    [java] at org.jpox.store.rdbms.request.FetchRequest.execute(FetchRequest.java:195)
    [java] at org.jpox.store.rdbms.table.ClassTable.fetch(ClassTable.java:1739)
    [java] at org.jpox.store.StoreManager.fetch(StoreManager.java:665)
    [java] at org.jpox.state.StateManagerImpl.loadDFGFields(StateManagerImpl.java:1573)
    [java] at org.jpox.state.StateManagerImpl.loadDefaultFetchGroup(StateManagerImpl.java:1666)
    [java] at org.jpox.state.StateManagerImpl.validate(StateManagerImpl.java:3456)
    [java] at org.jpox.AbstractPersistenceManager.getObjectById(AbstractPersistenceManager.java:2204)
    [java] at org.jpox.AbstractPersistenceManager.getObjectById(AbstractPersistenceManager.java:2107)
    [java] at org.apache.jdo.tck.api.persistencemanager.OptimisticFailure.runTestOptimistic(OptimisticFailure.java:139)
    [java] at org.apache.jdo.tck.api.persistencemanager.OptimisticFailure.test(OptimisticFailure.java:83)
    [java] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    [java] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    [java] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    [java] at java.lang.reflect.Method.invoke(Method.java:324)
    [java] at junit.framework.TestCase.runTest(TestCase.java:154)
    [java] at org.apache.jdo.tck.JDO_Test.runBare(JDO_Test.java:181)
    [java] at junit.framework.TestResult$1.protect(TestResult.java:106)
    [java] at junit.framework.TestResult.runProtected(TestResult.java:124)
    [java] at junit.framework.TestResult.run(TestResult.java:109)
    [java] at junit.framework.TestCase.run(TestCase.java:118)
    [java] at junit.framework.TestSuite.runTest(TestSuite.java:208)
    [java] at junit.framework.TestSuite.run(TestSuite.java:203)
    [java] at junit.framework.TestSuite.runTest(TestSuite.java:208)
    [java] at junit.framework.TestSuite.run(TestSuite.java:203)
    [java] at junit.textui.TestRunner.doRun(TestRunner.java:116)
    [java] at junit.textui.TestRunner.doRun(TestRunner.java:109)
    [java] at org.apache.jdo.tck.util.BatchTestRunner.start(BatchTestRunner.java:128)
    [java] at org.apache.jdo.tck.util.BatchTestRunner.main(BatchTestRunner.java:106)
    [java] NestedThrowablesStackTrace:
    [java] ERROR 40XL1: A lock could not be obtained within the time requested
    [java] at org.apache.derby.iapi.error.StandardException.newException(StandardException.java)
    [java] at org.apache.derby.impl.services.locks.LockSet.lockObject(LockSet.java)
    [java] at org.apache.derby.impl.services.locks.SinglePool.lockAnObject(SinglePool.java)
    [java] at org.apache.derby.impl.services.locks.SinglePool.lockObject(SinglePool.java)
    [java] at org.apache.derby.impl.store.raw.xact.RowLocking2.lockRecordForRead(RowLocking2.java)
    [java] at org.apache.derby.impl.store.access.heap.HeapController.lockRow(HeapController.java)
    [java] at org.apache.derby.impl.store.access.heap.HeapController.lockRow(HeapController.java)
    [java] at org.apache.derby.impl.store.access.btree.index.B2IRowLocking3.lockRowOnPage(B2IRowLocking3.java)
    [java] at org.apache.derby.impl.store.access.btree.index.B2IRowLocking3._lockScanRow(B2IRowLocking3.java)
    [java] at org.apache.derby.impl.store.access.btree.index.B2IRowLockingRR.lockScanRow(B2IRowLockingRR.java)
    [java] at org.apache.derby.impl.store.access.btree.BTreeForwardScan.fetchRows(BTreeForwardScan.java)
    [java] at org.apache.derby.impl.store.access.btree.BTreeScan.fetchNext(BTreeScan.java)
    [java] at org.apache.derby.impl.sql.execute.TableScanResultSet.getNextRowCore(TableScanResultSet.java)
    [java] at org.apache.derby.impl.sql.execute.IndexRowToBaseRowResultSet.getNextRowCore(IndexRowToBaseRowResultSet.java)
    [java] at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(ProjectRestrictResultSet.java)
    [java] at org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(BasicNoPutResultSetImpl.java)
    [java] at org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(EmbedResultSet.java)
    [java] at org.apache.derby.impl.jdbc.EmbedResultSet.next(EmbedResultSet.java)
    [java] at org.jpox.store.rdbms.request.FetchRequest.execute(FetchRequest.java:157)
    [java] at org.jpox.store.rdbms.table.ClassTable.fetch(ClassTable.java:1739)
    [java] at org.jpox.store.StoreManager.fetch(StoreManager.java:665)
    [java] at org.jpox.state.StateManagerImpl.loadDFGFields(StateManagerImpl.java:1573)
    [java] at org.jpox.state.StateManagerImpl.loadDefaultFetchGroup(StateManagerImpl.java:1666)
    [java] at org.jpox.state.StateManagerImpl.validate(StateManagerImpl.java:3456)
    [java] at org.jpox.AbstractPersistenceManager.getObjectById(AbstractPersistenceManager.java:2204)
    [java] at org.jpox.AbstractPersistenceManager.getObjectById(AbstractPersistenceManager.java:2107)
    [java] at org.apache.jdo.tck.api.persistencemanager.OptimisticFailure.runTestOptimistic(OptimisticFailure.java:139)
    [java] at org.apache.jdo.tck.api.persistencemanager.OptimisticFailure.test(OptimisticFailure.java:83)
    [java] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    [java] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    [java] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    [java] at java.lang.reflect.Method.invoke(Method.java:324)
    [java] at junit.framework.TestCase.runTest(TestCase.java:154)
    [java] at org.apache.jdo.tck.JDO_Test.runBare(JDO_Test.java:181)
    [java] at junit.framework.TestResult$1.protect(TestResult.java:106)
    [java] at junit.framework.TestResult.runProtected(TestResult.java:124)
    [java] at junit.framework.TestResult.run(TestResult.java:109)
    [java] at junit.framework.TestCase.run(TestCase.java:118)
    [java] at junit.framework.TestSuite.runTest(TestSuite.java:208)
    [java] at junit.framework.TestSuite.run(TestSuite.java:203)
    [java] at junit.framework.TestSuite.runTest(TestSuite.java:208)
    [java] at junit.framework.TestSuite.run(TestSuite.java:203)
    [java] at junit.textui.TestRunner.doRun(TestRunner.java:116)
    [java] at junit.textui.TestRunner.doRun(TestRunner.java:109)
    [java] at org.apache.jdo.tck.util.BatchTestRunner.start(BatchTestRunner.java:128)
    [java] at org.apache.jdo.tck.util.BatchTestRunner.main(BatchTestRunner.java:106)

Comments

[Andy Jefferson]

Looking at this test, it no longer hangs (so presumably someone changed something since it was reported) but contains something I'd like clarification on. It uses PCPoint objects, and expects version checking of the objects to fail. I look at the MetaData for PCPoint and see no <version> tags (and no version column in the schema either). Chapter 18 of the spec doesn't mention a default for version strategy, and so I'm questioning what the spec is expecting in terms of implementation behaviour. Is the JDO impl supposed to check all fields of all objects enlisted in an optimistic txn when there are no version tags ? (i.e "state-image" is the default version strategy) or is there something else that I'm missing here ?

[Erik Bengtson]

> it no longer hangs

I had a fix on JPOX. Now the test fails because it asserts that all DIRTY objects should have been checked and included in the OptimisticVerification exception raised, but JPOX stops at the first failed object.

[Craig Russell]

It appears that good progress is being made on the bug...

This is the relevant part of the specification from 13.5.

<spec>
If any instance fails the verification, a JDOOptimisticVerificationException is thrown which contains an array of JDOOptimisticVerificationException, one for each instance that failed the verification. The optimistic transaction is failed, and the transaction is rolled back. The definition of "changed instance" is a JDO implementation choice, but it is required that a field that has been changed to different values in different transactions results in one of the transactions failing.
</spec>

It is a requirement both that conflicting changes be recognized by the JDO implementation, and that all instances that fail the conflict check are reported in the exception.

In JDO 2, we added mapping metadata to specify versioning as a strategy. Without versioning strategy, if an implementation supports optimistic transactions it has to do some non-specified behavior to detect conflicting updates/deletes.

[Andy Jefferson]

You say that when no strategy is specifed it is for the JDO implementation to do some "non-specified behaviour to detect conflicting updates/deletes". This seemingly doesn't exclude adding a column to the datastore table to allow for detecting such changes. The only thing is, the TCK test does exclude this as a valid mechanism for detecting changes since it defines the schema and there is no available column for this. Wouldn't it be a better test to define an optimistic strategy and then there is no issue ?

[Craig Russell]

If you tell me what version strategs are supported by JPOX, I'll put the version information into the metadata. There are four strategies documented, all of them are required.

This might be a good opportunity for adding supportedOptions that tell which strategies are supported.

Current support is for "date-time", and "version-number".

[Craig Russell]

svn commit -m "JDO-72 added VersionedPCPoint with version column and version-number strategy; changed OptimisticFailure to use versioned class"
Sending tck20/project.properties
Sending tck20/test/java/org/apache/jdo/tck/api/persistencemanager/OptimisticFailure.java
Adding tck20/test/java/org/apache/jdo/tck/pc/mylib/VersionedPCPoint.java
Sending tck20/test/jdo/applicationidentity/org/apache/jdo/tck/pc/mylib/package.jdo
Sending tck20/test/jdo/datastoreidentity/org/apache/jdo/tck/pc/mylib/package.jdo
Sending tck20/test/orm/applicationidentity/org/apache/jdo/tck/pc/mylib/package-derby.orm
Sending tck20/test/orm/datastoreidentity/org/apache/jdo/tck/pc/mylib/package-derby.orm
Sending tck20/test/sql/derby/applicationidentity/schema.sql
Sending tck20/test/sql/derby/datastoreidentity/schema.sql
Transmitting file data .........
Committed revision 359415.

This change adds VersionedPCPoint with version columns and modifies the OptimisticFailure test to use it.

The test now encounters an optimistic commit failure, but incorrectly reports the failed instances. Instead of an array with the failed instance exceptions in it, there are no nested exceptions.

"Nested exceptions[] wrong size: expected 3, got 0" for both application and datastore identity.

[Andy Jefferson]

Fixed in JPOX CVS - builds dated 29/12/2005 or later.
Now returns 3 nested exceptions, and the test passes :-)