OpenJPA
  1. OpenJPA
  2. OPENJPA-466

Primary key constraint violated using (Oracle) sequence to generate ID in multithreaded app

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 1.0.0, 1.0.1, 1.1.0, 1.2.0
    • Fix Version/s: 1.0.4, 1.2.2, 1.3.0, 2.0.0-M3
    • Component/s: None
    • Labels:
      None
    • Environment:
      OpenJPA 1.0.0 (also tried 1.0.1 and 1.1.0-SNAPSHOT)
      Oracle XE 10g (JDBC driver 10.2.0.3.0)
      Windows XP Pro
    • Patch Info:
      Patch Available

      Description

      Here's how I annotate the ID:
      @Id
      @SequenceGenerator(name = "FooSeq", sequenceName = "seq_foo", allocationSize = 20)
      @GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "FooSeq")
      private Long id;

      Here's how I create the (Oracle) sequence:
      CREATE SEQUENCE seq_foo START WITH 1 INCREMENT BY 1;

      I get a primary key unique constraint violated in a multithreaded app i.e. it doesn't happen in single-threaded!

      You can simply reproduce this error by either create blocking queue or blocking thread pool say size 5 to insert 10000+ object.

      1. GeneratedIdObject.java
        0.5 kB
        Pinaki Poddar
      2. OPENJPA-466.patch
        1 kB
        Milosz Tylenda
      3. OPENJPA-466.patch
        29 kB
        Tim McConnell
      4. OPENJPA-466-1.0.x.patch
        30 kB
        B.J. Reed
      5. OPENJPA-466-1.2.x.patch
        30 kB
        B.J. Reed
      6. OPENJPA-466-SYNCRONIZED.patch
        0.7 kB
        Tim McConnell
      7. TestSequenceGenerationOnMT.java
        2 kB
        Pinaki Poddar
      8. volatile.patch
        0.6 kB
        Milosz Tylenda

        Issue Links

          Activity

          Hide
          Gil Markham added a comment -

          I've seen the same behavior using postgresql 8.2/8.3 and sequences and openjpa 1.0.1 and 1.0.2.

          Show
          Gil Markham added a comment - I've seen the same behavior using postgresql 8.2/8.3 and sequences and openjpa 1.0.1 and 1.0.2.
          Hide
          Joe Pullen added a comment -

          is this issue resolved ? its pretty old and I am experiencing a similar error with version 1.2.0 and the priority is Blocker ?

          Show
          Joe Pullen added a comment - is this issue resolved ? its pretty old and I am experiencing a similar error with version 1.2.0 and the priority is Blocker ?
          Hide
          Tim McConnell added a comment -

          Hi Joe, I'm trying to replicate the failure so that I can fix this problem. However, I'm still unable to reproduce it. Are you only seeing it fail in a multi-threaded application ?? And if so, are both threads connected to the same database and/or using the same sequence ?? Thanks for any additional information.

          Show
          Tim McConnell added a comment - Hi Joe, I'm trying to replicate the failure so that I can fix this problem. However, I'm still unable to reproduce it. Are you only seeing it fail in a multi-threaded application ?? And if so, are both threads connected to the same database and/or using the same sequence ?? Thanks for any additional information.
          Hide
          Joe Pullen added a comment -

          Hi Tim,

          Definately get the issue in a WebLogic multithreaded enviroment with 11g (RAC). The version of OpenJPA used is 1.2.0.
          The issues comes with more load (25-100 TPS). The application has about 25 entities all using seqs on the primary key.
          The error is always on the flush. The seq cache is set to 100.

          The test cases are not mutilthreaded in our app, but maybe I try to get a working, or is that non working test.
          Do you have ideas where the problem lies ?

          Show
          Joe Pullen added a comment - Hi Tim, Definately get the issue in a WebLogic multithreaded enviroment with 11g (RAC). The version of OpenJPA used is 1.2.0. The issues comes with more load (25-100 TPS). The application has about 25 entities all using seqs on the primary key. The error is always on the flush. The seq cache is set to 100. The test cases are not mutilthreaded in our app, but maybe I try to get a working, or is that non working test. Do you have ideas where the problem lies ?
          Hide
          Tim McConnell added a comment -

          Hi Joe, I think I'm finally able to replicate this error !! I haven't run it yet on Oracle, but I'm consistently able to get this failure using postgresql. Does this look similar to the error you're getting in Oracle ??

          Caused by: <openjpa-2.0.0-SNAPSHOT-r422266:742775 nonfatal general error> org.apache.openjpa.persistence.PersistenceException: ERROR: duplicate key value violates unique constraint "entity_employee_pkey"

          {prepstmnt 17832855 INSERT INTO ENTITY_EMPLOYEE (id, firstName, lastName, salary) VALUES (?, ?, ?, ?) [params=(int) 273350, (String) First_name_1628, (String) Last_name_1628, (float) 1628.0]}

          [code=0, state=23505]

          Show
          Tim McConnell added a comment - Hi Joe, I think I'm finally able to replicate this error !! I haven't run it yet on Oracle, but I'm consistently able to get this failure using postgresql. Does this look similar to the error you're getting in Oracle ?? Caused by: <openjpa-2.0.0-SNAPSHOT-r422266:742775 nonfatal general error> org.apache.openjpa.persistence.PersistenceException: ERROR: duplicate key value violates unique constraint "entity_employee_pkey" {prepstmnt 17832855 INSERT INTO ENTITY_EMPLOYEE (id, firstName, lastName, salary) VALUES (?, ?, ?, ?) [params=(int) 273350, (String) First_name_1628, (String) Last_name_1628, (float) 1628.0]} [code=0, state=23505]
          Hide
          Pinaki Poddar added a comment -

          Is openjpa.Multithreaded configuration property is set to 'true'?

          Show
          Pinaki Poddar added a comment - Is openjpa.Multithreaded configuration property is set to 'true'?
          Hide
          Tim McConnell added a comment -

          Hi Pinaki, in my postgresql testcase I'm setting the openjpa.Multithreaded property in the setup() mehtod as below. Does this look correct ??

          public void setUp()

          { setUp(EntityPerson.class, EntityEmployee.class, CLEAR_TABLES, "openjpa.Multithreaded", "true"); }
          Show
          Tim McConnell added a comment - Hi Pinaki, in my postgresql testcase I'm setting the openjpa.Multithreaded property in the setup() mehtod as below. Does this look correct ?? public void setUp() { setUp(EntityPerson.class, EntityEmployee.class, CLEAR_TABLES, "openjpa.Multithreaded", "true"); }
          Hide
          Joe Pullen added a comment - - edited

          Hi Tim,

          Looks similar to mine from oracle.

          ]]></stack><cause><stack><![CDATA[ <openjpa-1.2.0-r422266:683325 nonfatal general error> org.apache.openjpa.persistence.PersistenceException: ORA-00001: unique constraint (P
          OR.DSTA_PK) violated

          {prepstmnt 97 INSERT INTO DIA_STATUSES (DSTA_ID, APPLICATION_ID, COMPONENT_NAME, STATUS_LEVEL, REFERENCE_ID, SOFTWARE_VERSION, TIME_STAMP, VERSION, DATE_CREATED, CREATED_BY, DAT E_MODIFIED, MODIFIED_BY, PPPC_ID, PPRC_ID) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) [params=(long) 230503, (String) JTX, (String) input, (short) 1, (null) null, (String) 19-60-08, (Timestamp) 2009-02-10 17:15:05.49, (int) 1, (Timestamp) 2009-02-10 17:15:05.493, (String) weblogic, (Timestamp) 2009-02-10 17:15:05.493, (String) weblogic, (null) nul l, (null) null]}

          [code=1, state=23000]
          at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DBDictionary.java:4238)
          at org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(DBDictionary.java:4203)
          at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:102)
          at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:72)
          at org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushAndUpdate(PreparedStatementManagerImpl.java:131)
          at org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushInternal(PreparedStatementManagerImpl.java:89)
          at org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flush(PreparedStatementManagerImpl.java:72)
          at org.apache.openjpa.jdbc.kernel.OperationOrderUpdateManager.flushPrimaryRow(OperationOrderUpdateManager.java:203)
          at org.apache.openjpa.jdbc.kernel.OperationOrderUpdateManager.flush(OperationOrderUpdateManager.java:89)
          at org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdateManager.java:89)
          at org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdateManager.java:72)
          at org.apache.openjpa.jdbc.kernel.JDBCStoreManager.flush(JDBCStoreManager.java:655)

          Show
          Joe Pullen added a comment - - edited Hi Tim, Looks similar to mine from oracle. ]]></stack><cause><stack><![CDATA[ <openjpa-1.2.0-r422266:683325 nonfatal general error> org.apache.openjpa.persistence.PersistenceException: ORA-00001: unique constraint (P OR.DSTA_PK) violated {prepstmnt 97 INSERT INTO DIA_STATUSES (DSTA_ID, APPLICATION_ID, COMPONENT_NAME, STATUS_LEVEL, REFERENCE_ID, SOFTWARE_VERSION, TIME_STAMP, VERSION, DATE_CREATED, CREATED_BY, DAT E_MODIFIED, MODIFIED_BY, PPPC_ID, PPRC_ID) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) [params=(long) 230503, (String) JTX, (String) input, (short) 1, (null) null, (String) 19-60-08, (Timestamp) 2009-02-10 17:15:05.49, (int) 1, (Timestamp) 2009-02-10 17:15:05.493, (String) weblogic, (Timestamp) 2009-02-10 17:15:05.493, (String) weblogic, (null) nul l, (null) null]} [code=1, state=23000] at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DBDictionary.java:4238) at org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(DBDictionary.java:4203) at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:102) at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:72) at org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushAndUpdate(PreparedStatementManagerImpl.java:131) at org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushInternal(PreparedStatementManagerImpl.java:89) at org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flush(PreparedStatementManagerImpl.java:72) at org.apache.openjpa.jdbc.kernel.OperationOrderUpdateManager.flushPrimaryRow(OperationOrderUpdateManager.java:203) at org.apache.openjpa.jdbc.kernel.OperationOrderUpdateManager.flush(OperationOrderUpdateManager.java:89) at org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdateManager.java:89) at org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdateManager.java:72) at org.apache.openjpa.jdbc.kernel.JDBCStoreManager.flush(JDBCStoreManager.java:655)
          Hide
          Joe Pullen added a comment -

          tried with the openjpa.Multithreaded as true for oracle and didnt help

          Show
          Joe Pullen added a comment - tried with the openjpa.Multithreaded as true for oracle and didnt help
          Hide
          Tim McConnell added a comment -

          Hi again Joe, I can now replicate the problem on both postgresql and oracle databases. Here is the error I get on oracle. Now, I will determine the cause for each. Thanks for your help.....

          Caused by: <openjpa-2.0.0-SNAPSHOT-r422266:743437M nonfatal general error> org.apache.openjpa.persistence.PersistenceException: ORA-00001: unique constraint (SYSTEM.SYS_C004001) violated

          {prepstmnt 30673895 INSERT INTO ENTITY_PERSON (id, firstName, lastName) VALUES (?, ?, ?) [params=(int) 15662, (String) First_name_1011, (String) Last_name_1011]}

          [code=1, state=23000]
          FailedObject: org.apache.openjpa.persistence.hightps.sequence.EntityPerson@114c8b6
          at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DBDictionary.java:4244)
          at org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(DBDictionary.java:4209)
          at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:102)
          at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:72)
          at org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushAndUpdate(PreparedStatementManagerImpl.java:127)
          at org.apache.openjpa.jdbc.kernel.BatchingPreparedStatementManagerImpl.batchOrExecuteRow(BatchingPreparedStatementManagerImpl.java:100)
          at org.apache.openjpa.jdbc.kernel.BatchingPreparedStatementManagerImpl.flushAndUpdate(BatchingPreparedStatementManagerImpl.java:84)
          at org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushInternal(PreparedStatementManagerImpl.java:93)
          at org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flush(PreparedStatementManagerImpl.java:81)
          at org.apache.openjpa.jdbc.kernel.ConstraintUpdateManager.flush(ConstraintUpdateManager.java:549)
          at org.apache.openjpa.jdbc.kernel.ConstraintUpdateManager.flush(ConstraintUpdateManager.java:106)
          at org.apache.openjpa.jdbc.kernel.BatchingConstraintUpdateManager.flush(BatchingConstraintUpdateManager.java:59)
          at org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdateManager.java:89)
          at org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdateManager.java:72)
          at org.apache.openjpa.jdbc.kernel.JDBCStoreManager.flush(JDBCStoreManager.java:630)
          at org.apache.openjpa.kernel.DelegatingStoreManager.flush(DelegatingStoreManager.java:130)
          ... 15 more

          Show
          Tim McConnell added a comment - Hi again Joe, I can now replicate the problem on both postgresql and oracle databases. Here is the error I get on oracle. Now, I will determine the cause for each. Thanks for your help..... Caused by: <openjpa-2.0.0-SNAPSHOT-r422266:743437M nonfatal general error> org.apache.openjpa.persistence.PersistenceException: ORA-00001: unique constraint (SYSTEM.SYS_C004001) violated {prepstmnt 30673895 INSERT INTO ENTITY_PERSON (id, firstName, lastName) VALUES (?, ?, ?) [params=(int) 15662, (String) First_name_1011, (String) Last_name_1011]} [code=1, state=23000] FailedObject: org.apache.openjpa.persistence.hightps.sequence.EntityPerson@114c8b6 at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DBDictionary.java:4244) at org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(DBDictionary.java:4209) at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:102) at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:72) at org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushAndUpdate(PreparedStatementManagerImpl.java:127) at org.apache.openjpa.jdbc.kernel.BatchingPreparedStatementManagerImpl.batchOrExecuteRow(BatchingPreparedStatementManagerImpl.java:100) at org.apache.openjpa.jdbc.kernel.BatchingPreparedStatementManagerImpl.flushAndUpdate(BatchingPreparedStatementManagerImpl.java:84) at org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushInternal(PreparedStatementManagerImpl.java:93) at org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flush(PreparedStatementManagerImpl.java:81) at org.apache.openjpa.jdbc.kernel.ConstraintUpdateManager.flush(ConstraintUpdateManager.java:549) at org.apache.openjpa.jdbc.kernel.ConstraintUpdateManager.flush(ConstraintUpdateManager.java:106) at org.apache.openjpa.jdbc.kernel.BatchingConstraintUpdateManager.flush(BatchingConstraintUpdateManager.java:59) at org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdateManager.java:89) at org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdateManager.java:72) at org.apache.openjpa.jdbc.kernel.JDBCStoreManager.flush(JDBCStoreManager.java:630) at org.apache.openjpa.kernel.DelegatingStoreManager.flush(DelegatingStoreManager.java:130) ... 15 more
          Hide
          Tim McConnell added a comment -

          Failure for DB2 below. This was to demonstrate that it is not just a Oracle and PostgreSQL problem. There seems to be a thread-safe problem in OpenJPA for all databases that support native Sequences.

          testMultiThreadedLoad(org.apache.openjpa.persistence.hightps.sequence.TestSequence) Time elapsed: 178.797 sec <<< ERROR!
          org.apache.openjpa.persistence.hightps.sequence.TestSequence$ThreadingException: The 3 embedded errors occured in the execution of 8 iterations of 6 threads: [reflection invocation: (testMultiThreadedLoad)]
          at org.apache.openjpa.persistence.hightps.sequence.TestSequence.mttest(TestSequence.java:466)
          at org.apache.openjpa.persistence.hightps.sequence.TestSequence.mttest(TestSequence.java:360)
          at org.apache.openjpa.persistence.hightps.sequence.TestSequence.mttest(TestSequence.java:310)
          at org.apache.openjpa.persistence.hightps.sequence.TestSequence.mttest(TestSequence.java:304)
          at org.apache.openjpa.persistence.hightps.sequence.TestSequence.testMultiThreadedLoad(TestSequence.java:255)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          at java.lang.reflect.Method.invoke(Method.java:585)
          at junit.framework.TestCase.runTest(TestCase.java:154)
          at junit.framework.TestCase.runBare(TestCase.java:127)
          at org.apache.openjpa.persistence.test.PersistenceTestCase.runBare(PersistenceTestCase.java:443)
          at junit.framework.TestResult$1.protect(TestResult.java:106)
          at junit.framework.TestResult.runProtected(TestResult.java:124)
          at junit.framework.TestResult.run(TestResult.java:109)
          at junit.framework.TestCase.run(TestCase.java:118)
          at org.apache.openjpa.persistence.test.PersistenceTestCase.run(PersistenceTestCase.java:173)
          at junit.framework.TestSuite.runTest(TestSuite.java:208)
          at junit.framework.TestSuite.run(TestSuite.java:203)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          at java.lang.reflect.Method.invoke(Method.java:585)
          at org.apache.maven.surefire.junit.JUnitTestSet.execute(JUnitTestSet.java:213)
          at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSet(AbstractDirectoryTestSuite.java:140)
          at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute(AbstractDirectoryTestSuite.java:127)
          at org.apache.maven.surefire.Surefire.run(Surefire.java:177)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          at java.lang.reflect.Method.invoke(Method.java:585)
          at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:334)
          at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:980)
          Nested Throwable #1: org.apache.openjpa.persistence.hightps.sequence.TestSequence$ThreadingException: thread=Thread[reflection invocation: (testMultiThreadedLoad) [5 of 6],5,main];threadNum=5;maxThreads=6;iteration=1;maxIterations=8
          at org.apache.openjpa.persistence.hightps.sequence.TestSequence$2.run(TestSequence.java:429)
          Nested Throwable #1: java.lang.reflect.InvocationTargetException
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          at java.lang.reflect.Method.invoke(Method.java:585)
          at org.apache.openjpa.persistence.hightps.sequence.TestSequence$1.run(TestSequence.java:363)
          at org.apache.openjpa.persistence.hightps.sequence.TestSequence$2.run(TestSequence.java:422)
          Caused by: <openjpa-2.0.0-SNAPSHOT-r422266:743836M fatal store error> org.apache.openjpa.persistence.RollbackException: The transaction has been rolled back. See the nested exceptions for details on the errors that occurred.
          at org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImpl.java:530)
          at org.apache.openjpa.persistence.hightps.sequence.TestSequence.createManyPersonsInSeparateTransactions(TestSequence.java:134)
          at org.apache.openjpa.persistence.hightps.sequence.TestSequence.testMultiThreadedLoad(TestSequence.java:261)
          ... 6 more
          Caused by: <openjpa-2.0.0-SNAPSHOT-r422266:743836M fatal general error> org.apache.openjpa.persistence.PersistenceException: The transaction has been rolled back. See the nested exceptions for details on the errors that occurred.
          at org.apache.openjpa.kernel.BrokerImpl.newFlushException(BrokerImpl.java:2264)
          at org.apache.openjpa.kernel.BrokerImpl.flush(BrokerImpl.java:2111)
          at org.apache.openjpa.kernel.BrokerImpl.flushSafe(BrokerImpl.java:2009)
          at org.apache.openjpa.kernel.BrokerImpl.beforeCompletion(BrokerImpl.java:1927)
          at org.apache.openjpa.kernel.LocalManagedRuntime.commit(LocalManagedRuntime.java:81)
          at org.apache.openjpa.kernel.BrokerImpl.commit(BrokerImpl.java:1451)
          at org.apache.openjpa.kernel.DelegatingBroker.commit(DelegatingBroker.java:895)
          at org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImpl.java:519)
          ... 8 more
          Caused by: <openjpa-2.0.0-SNAPSHOT-r422266:743836M nonfatal general error> org.apache.openjpa.persistence.PersistenceException: One or more values in the INSERT statement, UPDATE statement, or foreign key update caused by a DELETE statement are not valid because the primary key, unique constraint or unique index identified by "1" constrains table "DB2ADMIN.ENTITY_PERSON" from having duplicate rows for those columns.
          FailedObject: prepstmnt 30057146 INSERT INTO ENTITY_PERSON (id, firstName, lastName) VALUES (?, ?, ?) [org.apache.openjpa.jdbc.kernel.JDBCStoreManager$CancelPreparedStatement]
          at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DBDictionary.java:4244)
          at org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(DBDictionary.java:4209)
          at org.apache.openjpa.jdbc.sql.DB2Dictionary.newStoreException(DB2Dictionary.java:507)
          at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:102)
          at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:72)
          at org.apache.openjpa.jdbc.kernel.BatchingPreparedStatementManagerImpl.flushBatch(BatchingPreparedStatementManagerImpl.java:193)
          at org.apache.openjpa.jdbc.kernel.BatchingConstraintUpdateManager.flush(BatchingConstraintUpdateManager.java:63)
          at org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdateManager.java:89)
          at org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdateManager.java:72)
          at org.apache.openjpa.jdbc.kernel.JDBCStoreManager.flush(JDBCStoreManager.java:630)
          at org.apache.openjpa.kernel.DelegatingStoreManager.flush(DelegatingStoreManager.java:130)
          ... 15 more
          Caused by: com.ibm.db2.jcc.b.SQLException: One or more values in the INSERT statement, UPDATE statement, or foreign key update caused by a DELETE statement are not valid because the primary key, unique constraint or unique index identified by "1" constrains table "DB2ADMIN.ENTITY_PERSON" from having duplicate rows for those columns.
          at com.ibm.db2.jcc.b.ce.d(ce.java:975)
          at com.ibm.db2.jcc.a.bd.k(bd.java:312)
          at com.ibm.db2.jcc.a.bd.a(bd.java:61)
          at com.ibm.db2.jcc.a.r.a(r.java:64)
          at com.ibm.db2.jcc.a.bq.c(bq.java:217)
          at com.ibm.db2.jcc.b.cf.C(cf.java:1109)
          at com.ibm.db2.jcc.b.cf.a(cf.java:1505)
          at com.ibm.db2.jcc.b.cf.executeUpdate(cf.java:322)
          at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:101)
          at org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:269)
          at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator$LoggingConnection$LoggingPreparedStatement.executeUpdate(LoggingConnectionDecorator.java:981)
          at org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:269)
          at org.apache.openjpa.jdbc.kernel.JDBCStoreManager$CancelPreparedStatement.executeUpdate(JDBCStoreManager.java:1501)
          at org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.executeUpdate(PreparedStatementManagerImpl.java:249)
          at org.apache.openjpa.jdbc.kernel.BatchingPreparedStatementManagerImpl.flushSingleRow(BatchingPreparedStatementManagerImpl.java:215)
          at org.apache.openjpa.jdbc.kernel.BatchingPreparedStatementManagerImpl.flushBatch(BatchingPreparedStatementManagerImpl.java:154)
          ... 20 more
          Nested Throwable #2: org.apache.openjpa.persistence.hightps.sequence.TestSequence$ThreadingException: thread=Thread[reflection invocation: (testMultiThreadedLoad) [2 of 6],5,main];threadNum=2;maxThreads=6;iteration=1;maxIterations=8
          at org.apache.openjpa.persistence.hightps.sequence.TestSequence$2.run(TestSequence.java:429)
          Nested Throwable #1: java.lang.reflect.InvocationTargetException
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          at java.lang.reflect.Method.invoke(Method.java:585)
          at org.apache.openjpa.persistence.hightps.sequence.TestSequence$1.run(TestSequence.java:363)
          at org.apache.openjpa.persistence.hightps.sequence.TestSequence$2.run(TestSequence.java:422)
          Caused by: <openjpa-2.0.0-SNAPSHOT-r422266:743836M fatal store error> org.apache.openjpa.persistence.RollbackException: The transaction has been rolled back. See the nested exceptions for details on the errors that occurred.
          at org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImpl.java:530)
          at org.apache.openjpa.persistence.hightps.sequence.TestSequence.createManyPersonsAndEmployeesInSeparateTransactions(TestSequence.java:179)
          at org.apache.openjpa.persistence.hightps.sequence.TestSequence.testMultiThreadedLoad(TestSequence.java:267)
          ... 6 more
          Caused by: <openjpa-2.0.0-SNAPSHOT-r422266:743836M fatal general error> org.apache.openjpa.persistence.PersistenceException: The transaction has been rolled back. See the nested exceptions for details on the errors that occurred.
          at org.apache.openjpa.kernel.BrokerImpl.newFlushException(BrokerImpl.java:2264)
          at org.apache.openjpa.kernel.BrokerImpl.flush(BrokerImpl.java:2111)
          at org.apache.openjpa.kernel.BrokerImpl.flushSafe(BrokerImpl.java:2009)
          at org.apache.openjpa.kernel.BrokerImpl.beforeCompletion(BrokerImpl.java:1927)
          at org.apache.openjpa.kernel.LocalManagedRuntime.commit(LocalManagedRuntime.java:81)
          at org.apache.openjpa.kernel.BrokerImpl.commit(BrokerImpl.java:1451)
          at org.apache.openjpa.kernel.DelegatingBroker.commit(DelegatingBroker.java:895)
          at org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImpl.java:519)
          ... 8 more
          Caused by: <openjpa-2.0.0-SNAPSHOT-r422266:743836M nonfatal general error> org.apache.openjpa.persistence.PersistenceException: The current transaction has been rolled back because of a deadlock or timeout. Reason code "2".
          FailedObject: prepstmnt 5148820 INSERT INTO ENTITY_EMPLOYEE (id, firstName, lastName, salary) VALUES (?, ?, ?, ?) [org.apache.openjpa.jdbc.kernel.JDBCStoreManager$CancelPreparedStatement]
          at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DBDictionary.java:4244)
          at org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(DBDictionary.java:4209)
          at org.apache.openjpa.jdbc.sql.DB2Dictionary.newStoreException(DB2Dictionary.java:507)
          at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:102)
          at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:72)
          at org.apache.openjpa.jdbc.kernel.BatchingPreparedStatementManagerImpl.flushBatch(BatchingPreparedStatementManagerImpl.java:193)
          at org.apache.openjpa.jdbc.kernel.BatchingConstraintUpdateManager.flush(BatchingConstraintUpdateManager.java:63)
          at org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdateManager.java:89)
          at org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdateManager.java:72)
          at org.apache.openjpa.jdbc.kernel.JDBCStoreManager.flush(JDBCStoreManager.java:630)
          at org.apache.openjpa.kernel.DelegatingStoreManager.flush(DelegatingStoreManager.java:130)
          ... 15 more
          Caused by: com.ibm.db2.jcc.b.SQLException: The current transaction has been rolled back because of a deadlock or timeout. Reason code "2".
          at com.ibm.db2.jcc.b.ce.e(ce.java:1093)
          at com.ibm.db2.jcc.a.bd.s(bd.java:678)
          at com.ibm.db2.jcc.a.bd.k(bd.java:335)
          at com.ibm.db2.jcc.a.bd.a(bd.java:61)
          at com.ibm.db2.jcc.a.r.a(r.java:64)
          at com.ibm.db2.jcc.a.bq.c(bq.java:217)
          at com.ibm.db2.jcc.b.cf.C(cf.java:1109)
          at com.ibm.db2.jcc.b.cf.a(cf.java:1505)
          at com.ibm.db2.jcc.b.cf.executeUpdate(cf.java:322)
          at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:101)
          at org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:269)
          at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator$LoggingConnection$LoggingPreparedStatement.executeUpdate(LoggingConnectionDecorator.java:981)
          at org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:269)
          at org.apache.openjpa.jdbc.kernel.JDBCStoreManager$CancelPreparedStatement.executeUpdate(JDBCStoreManager.java:1501)
          at org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.executeUpdate(PreparedStatementManagerImpl.java:249)
          at org.apache.openjpa.jdbc.kernel.BatchingPreparedStatementManagerImpl.flushSingleRow(BatchingPreparedStatementManagerImpl.java:215)
          at org.apache.openjpa.jdbc.kernel.BatchingPreparedStatementManagerImpl.flushBatch(BatchingPreparedStatementManagerImpl.java:154)
          ... 20 more
          Nested Throwable #3: org.apache.openjpa.persistence.hightps.sequence.TestSequence$ThreadingException: thread=Thread[reflection invocation: (testMultiThreadedLoad) [4 of 6],5,main];threadNum=4;maxThreads=6;iteration=1;maxIterations=8
          at org.apache.openjpa.persistence.hightps.sequence.TestSequence$2.run(TestSequence.java:429)
          Nested Throwable #1: java.lang.reflect.InvocationTargetException
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          at java.lang.reflect.Method.invoke(Method.java:585)
          at org.apache.openjpa.persistence.hightps.sequence.TestSequence$1.run(TestSequence.java:363)
          at org.apache.openjpa.persistence.hightps.sequence.TestSequence$2.run(TestSequence.java:422)
          Caused by: <openjpa-2.0.0-SNAPSHOT-r422266:743836M fatal store error> org.apache.openjpa.persistence.RollbackException: The transaction has been rolled back. See the nested exceptions for details on the errors that occurred.
          at org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImpl.java:530)
          at org.apache.openjpa.persistence.hightps.sequence.TestSequence.createManyPersonsAndEmployeesInSeparateTransactions(TestSequence.java:179)
          at org.apache.openjpa.persistence.hightps.sequence.TestSequence.testMultiThreadedLoad(TestSequence.java:267)
          ... 6 more
          Caused by: <openjpa-2.0.0-SNAPSHOT-r422266:743836M fatal general error> org.apache.openjpa.persistence.PersistenceException: The transaction has been rolled back. See the nested exceptions for details on the errors that occurred.
          at org.apache.openjpa.kernel.BrokerImpl.newFlushException(BrokerImpl.java:2264)
          at org.apache.openjpa.kernel.BrokerImpl.flush(BrokerImpl.java:2111)
          at org.apache.openjpa.kernel.BrokerImpl.flushSafe(BrokerImpl.java:2009)
          at org.apache.openjpa.kernel.BrokerImpl.beforeCompletion(BrokerImpl.java:1927)
          at org.apache.openjpa.kernel.LocalManagedRuntime.commit(LocalManagedRuntime.java:81)
          at org.apache.openjpa.kernel.BrokerImpl.commit(BrokerImpl.java:1451)
          at org.apache.openjpa.kernel.DelegatingBroker.commit(DelegatingBroker.java:895)
          at org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImpl.java:519)
          ... 8 more
          Caused by: <openjpa-2.0.0-SNAPSHOT-r422266:743836M nonfatal general error> org.apache.openjpa.persistence.PersistenceException: The current transaction has been rolled back because of a deadlock or timeout. Reason code "2".
          FailedObject: prepstmnt 6639365 INSERT INTO ENTITY_EMPLOYEE (id, firstName, lastName, salary) VALUES (?, ?, ?, ?) [org.apache.openjpa.jdbc.kernel.JDBCStoreManager$CancelPreparedStatement]
          at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DBDictionary.java:4244)
          at org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(DBDictionary.java:4209)
          at org.apache.openjpa.jdbc.sql.DB2Dictionary.newStoreException(DB2Dictionary.java:507)
          at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:102)
          at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:72)
          at org.apache.openjpa.jdbc.kernel.BatchingPreparedStatementManagerImpl.flushBatch(BatchingPreparedStatementManagerImpl.java:193)
          at org.apache.openjpa.jdbc.kernel.BatchingConstraintUpdateManager.flush(BatchingConstraintUpdateManager.java:63)
          at org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdateManager.java:89)
          at org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdateManager.java:72)
          at org.apache.openjpa.jdbc.kernel.JDBCStoreManager.flush(JDBCStoreManager.java:630)
          at org.apache.openjpa.kernel.DelegatingStoreManager.flush(DelegatingStoreManager.java:130)
          ... 15 more
          Caused by: com.ibm.db2.jcc.b.SQLException: The current transaction has been rolled back because of a deadlock or timeout. Reason code "2".
          at com.ibm.db2.jcc.b.ce.e(ce.java:1093)
          at com.ibm.db2.jcc.a.bd.s(bd.java:678)
          at com.ibm.db2.jcc.a.bd.k(bd.java:335)
          at com.ibm.db2.jcc.a.bd.a(bd.java:61)
          at com.ibm.db2.jcc.a.r.a(r.java:64)
          at com.ibm.db2.jcc.a.bq.c(bq.java:217)
          at com.ibm.db2.jcc.b.cf.C(cf.java:1109)
          at com.ibm.db2.jcc.b.cf.a(cf.java:1505)
          at com.ibm.db2.jcc.b.cf.executeUpdate(cf.java:322)
          at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:101)
          at org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:269)
          at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator$LoggingConnection$LoggingPreparedStatement.executeUpdate(LoggingConnectionDecorator.java:981)
          at org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:269)
          at org.apache.openjpa.jdbc.kernel.JDBCStoreManager$CancelPreparedStatement.executeUpdate(JDBCStoreManager.java:1501)
          at org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.executeUpdate(PreparedStatementManagerImpl.java:249)
          at org.apache.openjpa.jdbc.kernel.BatchingPreparedStatementManagerImpl.flushSingleRow(BatchingPreparedStatementManagerImpl.java:215)
          at org.apache.openjpa.jdbc.kernel.BatchingPreparedStatementManagerImpl.flushBatch(BatchingPreparedStatementManagerImpl.java:154)
          ... 20 more

          Show
          Tim McConnell added a comment - Failure for DB2 below. This was to demonstrate that it is not just a Oracle and PostgreSQL problem. There seems to be a thread-safe problem in OpenJPA for all databases that support native Sequences. testMultiThreadedLoad(org.apache.openjpa.persistence.hightps.sequence.TestSequence) Time elapsed: 178.797 sec <<< ERROR! org.apache.openjpa.persistence.hightps.sequence.TestSequence$ThreadingException: The 3 embedded errors occured in the execution of 8 iterations of 6 threads: [reflection invocation: (testMultiThreadedLoad)] at org.apache.openjpa.persistence.hightps.sequence.TestSequence.mttest(TestSequence.java:466) at org.apache.openjpa.persistence.hightps.sequence.TestSequence.mttest(TestSequence.java:360) at org.apache.openjpa.persistence.hightps.sequence.TestSequence.mttest(TestSequence.java:310) at org.apache.openjpa.persistence.hightps.sequence.TestSequence.mttest(TestSequence.java:304) at org.apache.openjpa.persistence.hightps.sequence.TestSequence.testMultiThreadedLoad(TestSequence.java:255) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at junit.framework.TestCase.runTest(TestCase.java:154) at junit.framework.TestCase.runBare(TestCase.java:127) at org.apache.openjpa.persistence.test.PersistenceTestCase.runBare(PersistenceTestCase.java:443) at junit.framework.TestResult$1.protect(TestResult.java:106) at junit.framework.TestResult.runProtected(TestResult.java:124) at junit.framework.TestResult.run(TestResult.java:109) at junit.framework.TestCase.run(TestCase.java:118) at org.apache.openjpa.persistence.test.PersistenceTestCase.run(PersistenceTestCase.java:173) at junit.framework.TestSuite.runTest(TestSuite.java:208) at junit.framework.TestSuite.run(TestSuite.java:203) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.apache.maven.surefire.junit.JUnitTestSet.execute(JUnitTestSet.java:213) at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSet(AbstractDirectoryTestSuite.java:140) at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute(AbstractDirectoryTestSuite.java:127) at org.apache.maven.surefire.Surefire.run(Surefire.java:177) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:334) at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:980) Nested Throwable #1: org.apache.openjpa.persistence.hightps.sequence.TestSequence$ThreadingException: thread=Thread[reflection invocation: (testMultiThreadedLoad) [5 of 6] ,5,main];threadNum=5;maxThreads=6;iteration=1;maxIterations=8 at org.apache.openjpa.persistence.hightps.sequence.TestSequence$2.run(TestSequence.java:429) Nested Throwable #1: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.apache.openjpa.persistence.hightps.sequence.TestSequence$1.run(TestSequence.java:363) at org.apache.openjpa.persistence.hightps.sequence.TestSequence$2.run(TestSequence.java:422) Caused by: <openjpa-2.0.0-SNAPSHOT-r422266:743836M fatal store error> org.apache.openjpa.persistence.RollbackException: The transaction has been rolled back. See the nested exceptions for details on the errors that occurred. at org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImpl.java:530) at org.apache.openjpa.persistence.hightps.sequence.TestSequence.createManyPersonsInSeparateTransactions(TestSequence.java:134) at org.apache.openjpa.persistence.hightps.sequence.TestSequence.testMultiThreadedLoad(TestSequence.java:261) ... 6 more Caused by: <openjpa-2.0.0-SNAPSHOT-r422266:743836M fatal general error> org.apache.openjpa.persistence.PersistenceException: The transaction has been rolled back. See the nested exceptions for details on the errors that occurred. at org.apache.openjpa.kernel.BrokerImpl.newFlushException(BrokerImpl.java:2264) at org.apache.openjpa.kernel.BrokerImpl.flush(BrokerImpl.java:2111) at org.apache.openjpa.kernel.BrokerImpl.flushSafe(BrokerImpl.java:2009) at org.apache.openjpa.kernel.BrokerImpl.beforeCompletion(BrokerImpl.java:1927) at org.apache.openjpa.kernel.LocalManagedRuntime.commit(LocalManagedRuntime.java:81) at org.apache.openjpa.kernel.BrokerImpl.commit(BrokerImpl.java:1451) at org.apache.openjpa.kernel.DelegatingBroker.commit(DelegatingBroker.java:895) at org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImpl.java:519) ... 8 more Caused by: <openjpa-2.0.0-SNAPSHOT-r422266:743836M nonfatal general error> org.apache.openjpa.persistence.PersistenceException: One or more values in the INSERT statement, UPDATE statement, or foreign key update caused by a DELETE statement are not valid because the primary key, unique constraint or unique index identified by "1" constrains table "DB2ADMIN.ENTITY_PERSON" from having duplicate rows for those columns. FailedObject: prepstmnt 30057146 INSERT INTO ENTITY_PERSON (id, firstName, lastName) VALUES (?, ?, ?) [org.apache.openjpa.jdbc.kernel.JDBCStoreManager$CancelPreparedStatement] at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DBDictionary.java:4244) at org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(DBDictionary.java:4209) at org.apache.openjpa.jdbc.sql.DB2Dictionary.newStoreException(DB2Dictionary.java:507) at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:102) at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:72) at org.apache.openjpa.jdbc.kernel.BatchingPreparedStatementManagerImpl.flushBatch(BatchingPreparedStatementManagerImpl.java:193) at org.apache.openjpa.jdbc.kernel.BatchingConstraintUpdateManager.flush(BatchingConstraintUpdateManager.java:63) at org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdateManager.java:89) at org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdateManager.java:72) at org.apache.openjpa.jdbc.kernel.JDBCStoreManager.flush(JDBCStoreManager.java:630) at org.apache.openjpa.kernel.DelegatingStoreManager.flush(DelegatingStoreManager.java:130) ... 15 more Caused by: com.ibm.db2.jcc.b.SQLException: One or more values in the INSERT statement, UPDATE statement, or foreign key update caused by a DELETE statement are not valid because the primary key, unique constraint or unique index identified by "1" constrains table "DB2ADMIN.ENTITY_PERSON" from having duplicate rows for those columns. at com.ibm.db2.jcc.b.ce.d(ce.java:975) at com.ibm.db2.jcc.a.bd.k(bd.java:312) at com.ibm.db2.jcc.a.bd.a(bd.java:61) at com.ibm.db2.jcc.a.r.a(r.java:64) at com.ibm.db2.jcc.a.bq.c(bq.java:217) at com.ibm.db2.jcc.b.cf.C(cf.java:1109) at com.ibm.db2.jcc.b.cf.a(cf.java:1505) at com.ibm.db2.jcc.b.cf.executeUpdate(cf.java:322) at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:101) at org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:269) at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator$LoggingConnection$LoggingPreparedStatement.executeUpdate(LoggingConnectionDecorator.java:981) at org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:269) at org.apache.openjpa.jdbc.kernel.JDBCStoreManager$CancelPreparedStatement.executeUpdate(JDBCStoreManager.java:1501) at org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.executeUpdate(PreparedStatementManagerImpl.java:249) at org.apache.openjpa.jdbc.kernel.BatchingPreparedStatementManagerImpl.flushSingleRow(BatchingPreparedStatementManagerImpl.java:215) at org.apache.openjpa.jdbc.kernel.BatchingPreparedStatementManagerImpl.flushBatch(BatchingPreparedStatementManagerImpl.java:154) ... 20 more Nested Throwable #2: org.apache.openjpa.persistence.hightps.sequence.TestSequence$ThreadingException: thread=Thread[reflection invocation: (testMultiThreadedLoad) [2 of 6] ,5,main];threadNum=2;maxThreads=6;iteration=1;maxIterations=8 at org.apache.openjpa.persistence.hightps.sequence.TestSequence$2.run(TestSequence.java:429) Nested Throwable #1: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.apache.openjpa.persistence.hightps.sequence.TestSequence$1.run(TestSequence.java:363) at org.apache.openjpa.persistence.hightps.sequence.TestSequence$2.run(TestSequence.java:422) Caused by: <openjpa-2.0.0-SNAPSHOT-r422266:743836M fatal store error> org.apache.openjpa.persistence.RollbackException: The transaction has been rolled back. See the nested exceptions for details on the errors that occurred. at org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImpl.java:530) at org.apache.openjpa.persistence.hightps.sequence.TestSequence.createManyPersonsAndEmployeesInSeparateTransactions(TestSequence.java:179) at org.apache.openjpa.persistence.hightps.sequence.TestSequence.testMultiThreadedLoad(TestSequence.java:267) ... 6 more Caused by: <openjpa-2.0.0-SNAPSHOT-r422266:743836M fatal general error> org.apache.openjpa.persistence.PersistenceException: The transaction has been rolled back. See the nested exceptions for details on the errors that occurred. at org.apache.openjpa.kernel.BrokerImpl.newFlushException(BrokerImpl.java:2264) at org.apache.openjpa.kernel.BrokerImpl.flush(BrokerImpl.java:2111) at org.apache.openjpa.kernel.BrokerImpl.flushSafe(BrokerImpl.java:2009) at org.apache.openjpa.kernel.BrokerImpl.beforeCompletion(BrokerImpl.java:1927) at org.apache.openjpa.kernel.LocalManagedRuntime.commit(LocalManagedRuntime.java:81) at org.apache.openjpa.kernel.BrokerImpl.commit(BrokerImpl.java:1451) at org.apache.openjpa.kernel.DelegatingBroker.commit(DelegatingBroker.java:895) at org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImpl.java:519) ... 8 more Caused by: <openjpa-2.0.0-SNAPSHOT-r422266:743836M nonfatal general error> org.apache.openjpa.persistence.PersistenceException: The current transaction has been rolled back because of a deadlock or timeout. Reason code "2". FailedObject: prepstmnt 5148820 INSERT INTO ENTITY_EMPLOYEE (id, firstName, lastName, salary) VALUES (?, ?, ?, ?) [org.apache.openjpa.jdbc.kernel.JDBCStoreManager$CancelPreparedStatement] at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DBDictionary.java:4244) at org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(DBDictionary.java:4209) at org.apache.openjpa.jdbc.sql.DB2Dictionary.newStoreException(DB2Dictionary.java:507) at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:102) at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:72) at org.apache.openjpa.jdbc.kernel.BatchingPreparedStatementManagerImpl.flushBatch(BatchingPreparedStatementManagerImpl.java:193) at org.apache.openjpa.jdbc.kernel.BatchingConstraintUpdateManager.flush(BatchingConstraintUpdateManager.java:63) at org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdateManager.java:89) at org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdateManager.java:72) at org.apache.openjpa.jdbc.kernel.JDBCStoreManager.flush(JDBCStoreManager.java:630) at org.apache.openjpa.kernel.DelegatingStoreManager.flush(DelegatingStoreManager.java:130) ... 15 more Caused by: com.ibm.db2.jcc.b.SQLException: The current transaction has been rolled back because of a deadlock or timeout. Reason code "2". at com.ibm.db2.jcc.b.ce.e(ce.java:1093) at com.ibm.db2.jcc.a.bd.s(bd.java:678) at com.ibm.db2.jcc.a.bd.k(bd.java:335) at com.ibm.db2.jcc.a.bd.a(bd.java:61) at com.ibm.db2.jcc.a.r.a(r.java:64) at com.ibm.db2.jcc.a.bq.c(bq.java:217) at com.ibm.db2.jcc.b.cf.C(cf.java:1109) at com.ibm.db2.jcc.b.cf.a(cf.java:1505) at com.ibm.db2.jcc.b.cf.executeUpdate(cf.java:322) at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:101) at org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:269) at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator$LoggingConnection$LoggingPreparedStatement.executeUpdate(LoggingConnectionDecorator.java:981) at org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:269) at org.apache.openjpa.jdbc.kernel.JDBCStoreManager$CancelPreparedStatement.executeUpdate(JDBCStoreManager.java:1501) at org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.executeUpdate(PreparedStatementManagerImpl.java:249) at org.apache.openjpa.jdbc.kernel.BatchingPreparedStatementManagerImpl.flushSingleRow(BatchingPreparedStatementManagerImpl.java:215) at org.apache.openjpa.jdbc.kernel.BatchingPreparedStatementManagerImpl.flushBatch(BatchingPreparedStatementManagerImpl.java:154) ... 20 more Nested Throwable #3: org.apache.openjpa.persistence.hightps.sequence.TestSequence$ThreadingException: thread=Thread[reflection invocation: (testMultiThreadedLoad) [4 of 6] ,5,main];threadNum=4;maxThreads=6;iteration=1;maxIterations=8 at org.apache.openjpa.persistence.hightps.sequence.TestSequence$2.run(TestSequence.java:429) Nested Throwable #1: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.apache.openjpa.persistence.hightps.sequence.TestSequence$1.run(TestSequence.java:363) at org.apache.openjpa.persistence.hightps.sequence.TestSequence$2.run(TestSequence.java:422) Caused by: <openjpa-2.0.0-SNAPSHOT-r422266:743836M fatal store error> org.apache.openjpa.persistence.RollbackException: The transaction has been rolled back. See the nested exceptions for details on the errors that occurred. at org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImpl.java:530) at org.apache.openjpa.persistence.hightps.sequence.TestSequence.createManyPersonsAndEmployeesInSeparateTransactions(TestSequence.java:179) at org.apache.openjpa.persistence.hightps.sequence.TestSequence.testMultiThreadedLoad(TestSequence.java:267) ... 6 more Caused by: <openjpa-2.0.0-SNAPSHOT-r422266:743836M fatal general error> org.apache.openjpa.persistence.PersistenceException: The transaction has been rolled back. See the nested exceptions for details on the errors that occurred. at org.apache.openjpa.kernel.BrokerImpl.newFlushException(BrokerImpl.java:2264) at org.apache.openjpa.kernel.BrokerImpl.flush(BrokerImpl.java:2111) at org.apache.openjpa.kernel.BrokerImpl.flushSafe(BrokerImpl.java:2009) at org.apache.openjpa.kernel.BrokerImpl.beforeCompletion(BrokerImpl.java:1927) at org.apache.openjpa.kernel.LocalManagedRuntime.commit(LocalManagedRuntime.java:81) at org.apache.openjpa.kernel.BrokerImpl.commit(BrokerImpl.java:1451) at org.apache.openjpa.kernel.DelegatingBroker.commit(DelegatingBroker.java:895) at org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImpl.java:519) ... 8 more Caused by: <openjpa-2.0.0-SNAPSHOT-r422266:743836M nonfatal general error> org.apache.openjpa.persistence.PersistenceException: The current transaction has been rolled back because of a deadlock or timeout. Reason code "2". FailedObject: prepstmnt 6639365 INSERT INTO ENTITY_EMPLOYEE (id, firstName, lastName, salary) VALUES (?, ?, ?, ?) [org.apache.openjpa.jdbc.kernel.JDBCStoreManager$CancelPreparedStatement] at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DBDictionary.java:4244) at org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(DBDictionary.java:4209) at org.apache.openjpa.jdbc.sql.DB2Dictionary.newStoreException(DB2Dictionary.java:507) at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:102) at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:72) at org.apache.openjpa.jdbc.kernel.BatchingPreparedStatementManagerImpl.flushBatch(BatchingPreparedStatementManagerImpl.java:193) at org.apache.openjpa.jdbc.kernel.BatchingConstraintUpdateManager.flush(BatchingConstraintUpdateManager.java:63) at org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdateManager.java:89) at org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdateManager.java:72) at org.apache.openjpa.jdbc.kernel.JDBCStoreManager.flush(JDBCStoreManager.java:630) at org.apache.openjpa.kernel.DelegatingStoreManager.flush(DelegatingStoreManager.java:130) ... 15 more Caused by: com.ibm.db2.jcc.b.SQLException: The current transaction has been rolled back because of a deadlock or timeout. Reason code "2". at com.ibm.db2.jcc.b.ce.e(ce.java:1093) at com.ibm.db2.jcc.a.bd.s(bd.java:678) at com.ibm.db2.jcc.a.bd.k(bd.java:335) at com.ibm.db2.jcc.a.bd.a(bd.java:61) at com.ibm.db2.jcc.a.r.a(r.java:64) at com.ibm.db2.jcc.a.bq.c(bq.java:217) at com.ibm.db2.jcc.b.cf.C(cf.java:1109) at com.ibm.db2.jcc.b.cf.a(cf.java:1505) at com.ibm.db2.jcc.b.cf.executeUpdate(cf.java:322) at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:101) at org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:269) at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator$LoggingConnection$LoggingPreparedStatement.executeUpdate(LoggingConnectionDecorator.java:981) at org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:269) at org.apache.openjpa.jdbc.kernel.JDBCStoreManager$CancelPreparedStatement.executeUpdate(JDBCStoreManager.java:1501) at org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.executeUpdate(PreparedStatementManagerImpl.java:249) at org.apache.openjpa.jdbc.kernel.BatchingPreparedStatementManagerImpl.flushSingleRow(BatchingPreparedStatementManagerImpl.java:215) at org.apache.openjpa.jdbc.kernel.BatchingPreparedStatementManagerImpl.flushBatch(BatchingPreparedStatementManagerImpl.java:154) ... 20 more
          Hide
          Tim McConnell added a comment -

          The attached patch provides a testcase to demonstrate the failure on both PostgreSQL and Oracle, and likewise provides a fix for both PostgreSQL and Oracle. I would suggest that it be applied to both Trunk and the 1.2.x branch. Thanks

          Show
          Tim McConnell added a comment - The attached patch provides a testcase to demonstrate the failure on both PostgreSQL and Oracle, and likewise provides a fix for both PostgreSQL and Oracle. I would suggest that it be applied to both Trunk and the 1.2.x branch. Thanks
          Hide
          Joe Pullen added a comment -

          Hi Tim, thanks for the patch I can guarantee to anyone who is using OpenJPA with a larger configuration (4 solaris machines with 24 core each) and oracle sequences that this patch is a MUST.
          After testing with the patch our problems with duplicate pk from sequences have disappeared. Also the patch doesnt seem to have a major impact on performance.

          Show
          Joe Pullen added a comment - Hi Tim, thanks for the patch I can guarantee to anyone who is using OpenJPA with a larger configuration (4 solaris machines with 24 core each) and oracle sequences that this patch is a MUST. After testing with the patch our problems with duplicate pk from sequences have disappeared. Also the patch doesnt seem to have a major impact on performance.
          Hide
          Tim McConnell added a comment -

          HI Joe, that's wonderful new !! Thanks much for verifying that it does it fact work for your configuration.....

          Show
          Tim McConnell added a comment - HI Joe, that's wonderful new !! Thanks much for verifying that it does it fact work for your configuration.....
          Hide
          Tim McConnell added a comment -

          Closing since we now have verification that it fixes the problem.

          Show
          Tim McConnell added a comment - Closing since we now have verification that it fixes the problem.
          Hide
          Milosz Tylenda added a comment -

          Although the patch applied (synchronization on PreparedStatement within NativeJDBCSeq.getSequence) does a good job on reducing the likelihood of crash, I am afraid we are still not free from multi-threading issue. On my one-core laptop I am not able to reproduce this issue (even when I remove the patch) but I find the problem still exists in the AbstractJDBCSeq.next method. I am able to reproduce the exception when I insert Thread.yield():

          public Object next(StoreContext ctx, ClassMetaData meta) {
          JDBCStore store = getStore(ctx);
          try

          { current = nextInternal(store, (ClassMapping) meta); Thread.yield(); return current; }

          catch (OpenJPAException ke)

          { throw ke; } catch (SQLException se) { throw SQLExceptions.getStore(se, store.getDBDictionary()); } catch (Exception e) { throw new StoreException(e); }
          }

          This yield() simulates context switching in an real application. The context switch will actually seldom occur here but is possible to my knowledge. Also, I am able to reproduce the exception by adding some Thread.sleep calls instead of yield() but this is harder to reproduce.

          My suggestion is to remove the synchronization from NativeJDBCSeq.getSequence and modify AbstractJDBCSeq.next to something like the following:

          public Object next(StoreContext ctx, ClassMetaData meta) {
          JDBCStore store = getStore(ctx);
          try { Object currentLocal = nextInternal(store, (ClassMapping) meta); current = currentLocal; return currentLocal; } catch (OpenJPAException ke) { throw ke; }

          catch (SQLException se)

          { throw SQLExceptions.getStore(se, store.getDBDictionary()); }

          catch (Exception e)

          { throw new StoreException(e); }

          }

          Show
          Milosz Tylenda added a comment - Although the patch applied (synchronization on PreparedStatement within NativeJDBCSeq.getSequence) does a good job on reducing the likelihood of crash, I am afraid we are still not free from multi-threading issue. On my one-core laptop I am not able to reproduce this issue (even when I remove the patch) but I find the problem still exists in the AbstractJDBCSeq.next method. I am able to reproduce the exception when I insert Thread.yield(): public Object next(StoreContext ctx, ClassMetaData meta) { JDBCStore store = getStore(ctx); try { current = nextInternal(store, (ClassMapping) meta); Thread.yield(); return current; } catch (OpenJPAException ke) { throw ke; } catch (SQLException se) { throw SQLExceptions.getStore(se, store.getDBDictionary()); } catch (Exception e) { throw new StoreException(e); } } This yield() simulates context switching in an real application. The context switch will actually seldom occur here but is possible to my knowledge. Also, I am able to reproduce the exception by adding some Thread.sleep calls instead of yield() but this is harder to reproduce. My suggestion is to remove the synchronization from NativeJDBCSeq.getSequence and modify AbstractJDBCSeq.next to something like the following: public Object next(StoreContext ctx, ClassMetaData meta) { JDBCStore store = getStore(ctx); try { Object currentLocal = nextInternal(store, (ClassMapping) meta); current = currentLocal; return currentLocal; } catch (OpenJPAException ke) { throw ke; } catch (SQLException se) { throw SQLExceptions.getStore(se, store.getDBDictionary()); } catch (Exception e) { throw new StoreException(e); } }
          Hide
          Milosz Tylenda added a comment -

          The attached patch implements my suggestion. I think this is a better solution than we currently have because:

          • the multi-threading problem will be completely avoided to my knowledge
          • by removing synchronization from PreparedStatement execution we gain a bit of scalability.

          I am going to commit it unless somebody has objection.

          Show
          Milosz Tylenda added a comment - The attached patch implements my suggestion. I think this is a better solution than we currently have because: the multi-threading problem will be completely avoided to my knowledge by removing synchronization from PreparedStatement execution we gain a bit of scalability. I am going to commit it unless somebody has objection.
          Hide
          Milosz Tylenda added a comment -

          I have applied the patch to trunk and 1.3.x branch.

          Show
          Milosz Tylenda added a comment - I have applied the patch to trunk and 1.3.x branch.
          Hide
          Tim McConnell added a comment -

          Hi Milosz, It's a little unclear to me how these changes will prevent duplicate sequence values. And actually, I can easily create duplicates with these changes in trunk using the postgreSQL database (by altering the number of entities created in the TestSequence testcase). See the error/exception messages below and look for "duplicate key value violates unique constraint", which is indicative of duplicate sequences getting created. This appears to be a regression as these exceptions were eliminated using the previous patch.....

          -------------------------------------------------------
          T E S T S
          -------------------------------------------------------
          Running org.apache.openjpa.persistence.sequence.TestSequence
          359 test INFO [main] openjpa.jdbc.JDBC - Using dictionary class "org.apache.openjpa.jdbc.sql.PostgresDictionary" (PostgreSQL 8.3.6 ,PostgreSQL Native Driver PostgreSQL 8.3 JDBC3 with SSL (build 603)).
          1406 test INFO [reflection invocation: (testMultiThreadedNativeSequences) [6 of 6]] openjpa.Runtime - Starting OpenJPA 2.0.0-SNAPSHOT
          Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 361.859 sec <<< FAILURE!
          testMultiThreadedNativeSequences(org.apache.openjpa.persistence.sequence.TestSequence) Time elapsed: 361.765 sec <<< ERROR!
          org.apache.openjpa.persistence.sequence.TestSequence$ThreadingException: The 1 embedded errors occured in the execution of 8 iterations of 6 threads: [reflection invocation: (testMultiThreadedNativeSequences)]
          at org.apache.openjpa.persistence.sequence.TestSequence.mttest(TestSequence.java:440)
          at org.apache.openjpa.persistence.sequence.TestSequence.mttest(TestSequence.java:334)
          at org.apache.openjpa.persistence.sequence.TestSequence.mttest(TestSequence.java:281)
          at org.apache.openjpa.persistence.sequence.TestSequence.mttest(TestSequence.java:275)
          at org.apache.openjpa.persistence.sequence.TestSequence.testMultiThreadedNativeSequences(TestSequence.java:66)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          at java.lang.reflect.Method.invoke(Method.java:585)
          at junit.framework.TestCase.runTest(TestCase.java:154)
          at junit.framework.TestCase.runBare(TestCase.java:127)
          at org.apache.openjpa.persistence.test.PersistenceTestCase.runBare(PersistenceTestCase.java:465)
          at junit.framework.TestResult$1.protect(TestResult.java:106)
          at junit.framework.TestResult.runProtected(TestResult.java:124)
          at junit.framework.TestResult.run(TestResult.java:109)
          at junit.framework.TestCase.run(TestCase.java:118)
          at org.apache.openjpa.persistence.test.PersistenceTestCase.run(PersistenceTestCase.java:181)
          at junit.framework.TestSuite.runTest(TestSuite.java:208)
          at junit.framework.TestSuite.run(TestSuite.java:203)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          at java.lang.reflect.Method.invoke(Method.java:585)
          at org.apache.maven.surefire.junit.JUnitTestSet.execute(JUnitTestSet.java:213)
          at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSet(AbstractDirectoryTestSuite.java:140)
          at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute(AbstractDirectoryTestSuite.java:127)
          at org.apache.maven.surefire.Surefire.run(Surefire.java:177)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          at java.lang.reflect.Method.invoke(Method.java:585)
          at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:345)
          at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1009)
          Nested Throwable #1: org.apache.openjpa.persistence.sequence.TestSequence$ThreadingException: thread=Thread[reflection invocation: (testMultiThreadedNativeSequences) [1 of 6],5,main];threadNum=1;maxThreads=6;iteration=2;maxIterations=8
          at org.apache.openjpa.persistence.sequence.TestSequence$2.run(TestSequence.java:405)
          Nested Throwable #1: java.lang.reflect.InvocationTargetException
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          at java.lang.reflect.Method.invoke(Method.java:585)
          at org.apache.openjpa.persistence.sequence.TestSequence$1.run(TestSequence.java:337)
          at org.apache.openjpa.persistence.sequence.TestSequence$2.run(TestSequence.java:398)
          Caused by: <openjpa-2.0.0-SNAPSHOT-r422266:774393M fatal store error> org.apache.openjpa.persistence.RollbackException: The transaction has been rolled back. See the nested exceptions for details on the errors that occurred.
          at org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImpl.java:559)
          at org.apache.openjpa.persistence.sequence.TestSequence.createManyPersonsAndEmployeesInSeparateTransactions(TestSequence.java:191)
          at org.apache.openjpa.persistence.sequence.TestSequence.testMultiThreadedNativeSequences(TestSequence.java:78)
          ... 6 more
          Caused by: <openjpa-2.0.0-SNAPSHOT-r422266:774393M fatal general error> org.apache.openjpa.persistence.PersistenceException: The transaction has been rolled back. See the nested exceptions for details on the errors that occurred.
          at org.apache.openjpa.kernel.BrokerImpl.newFlushException(BrokerImpl.java:2266)
          at org.apache.openjpa.kernel.BrokerImpl.flush(BrokerImpl.java:2113)
          at org.apache.openjpa.kernel.BrokerImpl.flushSafe(BrokerImpl.java:2011)
          at org.apache.openjpa.kernel.BrokerImpl.beforeCompletion(BrokerImpl.java:1929)
          at org.apache.openjpa.kernel.LocalManagedRuntime.commit(LocalManagedRuntime.java:81)
          at org.apache.openjpa.kernel.BrokerImpl.commit(BrokerImpl.java:1453)
          at org.apache.openjpa.kernel.DelegatingBroker.commit(DelegatingBroker.java:895)
          at org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImpl.java:548)
          ... 8 more
          Caused by: <openjpa-2.0.0-SNAPSHOT-r422266:774393M fatal general error> org.apache.openjpa.persistence.PersistenceException: ERROR: duplicate key value violates unique constraint "entity_employee_pkey"

          {prepstmnt 24724157 INSERT INTO ENTITY_EMPLOYEE (id, firstName, lastName, salary) VALUES (?, ?, ?, ?) [params=(int) 256631, (String) 4_First_name_6912, (String) 4_Last_name_6912, (float) 6912.0]} [code=0, state=23505]
          FailedObject: org.apache.openjpa.persistence.sequence.EntityEmployee@ff5a2b
          at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DBDictionary.java:4383)
          at org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(DBDictionary.java:4342)
          at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:102)
          at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:72)
          at org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushAndUpdate(PreparedStatementManagerImpl.java:141)
          at org.apache.openjpa.jdbc.kernel.BatchingPreparedStatementManagerImpl.flushAndUpdate(BatchingPreparedStatementManagerImpl.java:80)
          at org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushInternal(PreparedStatementManagerImpl.java:97)
          at org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flush(PreparedStatementManagerImpl.java:85)
          at org.apache.openjpa.jdbc.kernel.ConstraintUpdateManager.flush(ConstraintUpdateManager.java:550)
          at org.apache.openjpa.jdbc.kernel.ConstraintUpdateManager.flush(ConstraintUpdateManager.java:106)
          at org.apache.openjpa.jdbc.kernel.BatchingConstraintUpdateManager.flush(BatchingConstraintUpdateManager.java:59)
          at org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdateManager.java:103)
          at org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdateManager.java:76)
          at org.apache.openjpa.jdbc.kernel.JDBCStoreManager.flush(JDBCStoreManager.java:672)
          at org.apache.openjpa.kernel.DelegatingStoreManager.flush(DelegatingStoreManager.java:130)
          ... 15 more
          Caused by: org.apache.openjpa.lib.jdbc.ReportingSQLException: ERROR: duplicate key value violates unique constraint "entity_employee_pkey" {prepstmnt 24724157 INSERT INTO ENTITY_EMPLOYEE (id, firstName, lastName, salary) VALUES (?, ?, ?, ?) [params=(int) 256631, (String) 4_First_name_6912, (String) 4_Last_name_6912, (float) 6912.0]}

          [code=0, state=23505]
          at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(LoggingConnectionDecorator.java:236)
          at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.access$700(LoggingConnectionDecorator.java:69)
          at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator$LoggingConnection$LoggingPreparedStatement.executeUpdate(LoggingConnectionDecorator.java:1080)
          at org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:286)
          at org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:286)
          at org.apache.openjpa.jdbc.kernel.JDBCStoreManager$CancelPreparedStatement.executeUpdate(JDBCStoreManager.java:1579)
          at org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.executeUpdate(PreparedStatementManagerImpl.java:246)
          at org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushAndUpdate(PreparedStatementManagerImpl.java:116)
          ... 25 more

          Results :

          Tests in error:
          testMultiThreadedNativeSequences(org.apache.openjpa.persistence.sequence.TestSequence)

          Tests run: 1, Failures: 0, Errors: 1, Skipped: 0

          [INFO] ------------------------------------------------------------------------
          [ERROR] BUILD FAILURE
          [INFO] ------------------------------------------------------------------------
          [INFO] There are test failures.

          Show
          Tim McConnell added a comment - Hi Milosz, It's a little unclear to me how these changes will prevent duplicate sequence values. And actually, I can easily create duplicates with these changes in trunk using the postgreSQL database (by altering the number of entities created in the TestSequence testcase). See the error/exception messages below and look for "duplicate key value violates unique constraint", which is indicative of duplicate sequences getting created. This appears to be a regression as these exceptions were eliminated using the previous patch..... ------------------------------------------------------- T E S T S ------------------------------------------------------- Running org.apache.openjpa.persistence.sequence.TestSequence 359 test INFO [main] openjpa.jdbc.JDBC - Using dictionary class "org.apache.openjpa.jdbc.sql.PostgresDictionary" (PostgreSQL 8.3.6 ,PostgreSQL Native Driver PostgreSQL 8.3 JDBC3 with SSL (build 603)). 1406 test INFO [reflection invocation: (testMultiThreadedNativeSequences) [6 of 6] ] openjpa.Runtime - Starting OpenJPA 2.0.0-SNAPSHOT Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 361.859 sec <<< FAILURE! testMultiThreadedNativeSequences(org.apache.openjpa.persistence.sequence.TestSequence) Time elapsed: 361.765 sec <<< ERROR! org.apache.openjpa.persistence.sequence.TestSequence$ThreadingException: The 1 embedded errors occured in the execution of 8 iterations of 6 threads: [reflection invocation: (testMultiThreadedNativeSequences)] at org.apache.openjpa.persistence.sequence.TestSequence.mttest(TestSequence.java:440) at org.apache.openjpa.persistence.sequence.TestSequence.mttest(TestSequence.java:334) at org.apache.openjpa.persistence.sequence.TestSequence.mttest(TestSequence.java:281) at org.apache.openjpa.persistence.sequence.TestSequence.mttest(TestSequence.java:275) at org.apache.openjpa.persistence.sequence.TestSequence.testMultiThreadedNativeSequences(TestSequence.java:66) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at junit.framework.TestCase.runTest(TestCase.java:154) at junit.framework.TestCase.runBare(TestCase.java:127) at org.apache.openjpa.persistence.test.PersistenceTestCase.runBare(PersistenceTestCase.java:465) at junit.framework.TestResult$1.protect(TestResult.java:106) at junit.framework.TestResult.runProtected(TestResult.java:124) at junit.framework.TestResult.run(TestResult.java:109) at junit.framework.TestCase.run(TestCase.java:118) at org.apache.openjpa.persistence.test.PersistenceTestCase.run(PersistenceTestCase.java:181) at junit.framework.TestSuite.runTest(TestSuite.java:208) at junit.framework.TestSuite.run(TestSuite.java:203) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.apache.maven.surefire.junit.JUnitTestSet.execute(JUnitTestSet.java:213) at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSet(AbstractDirectoryTestSuite.java:140) at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute(AbstractDirectoryTestSuite.java:127) at org.apache.maven.surefire.Surefire.run(Surefire.java:177) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:345) at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1009) Nested Throwable #1: org.apache.openjpa.persistence.sequence.TestSequence$ThreadingException: thread=Thread[reflection invocation: (testMultiThreadedNativeSequences) [1 of 6] ,5,main];threadNum=1;maxThreads=6;iteration=2;maxIterations=8 at org.apache.openjpa.persistence.sequence.TestSequence$2.run(TestSequence.java:405) Nested Throwable #1: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.apache.openjpa.persistence.sequence.TestSequence$1.run(TestSequence.java:337) at org.apache.openjpa.persistence.sequence.TestSequence$2.run(TestSequence.java:398) Caused by: <openjpa-2.0.0-SNAPSHOT-r422266:774393M fatal store error> org.apache.openjpa.persistence.RollbackException: The transaction has been rolled back. See the nested exceptions for details on the errors that occurred. at org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImpl.java:559) at org.apache.openjpa.persistence.sequence.TestSequence.createManyPersonsAndEmployeesInSeparateTransactions(TestSequence.java:191) at org.apache.openjpa.persistence.sequence.TestSequence.testMultiThreadedNativeSequences(TestSequence.java:78) ... 6 more Caused by: <openjpa-2.0.0-SNAPSHOT-r422266:774393M fatal general error> org.apache.openjpa.persistence.PersistenceException: The transaction has been rolled back. See the nested exceptions for details on the errors that occurred. at org.apache.openjpa.kernel.BrokerImpl.newFlushException(BrokerImpl.java:2266) at org.apache.openjpa.kernel.BrokerImpl.flush(BrokerImpl.java:2113) at org.apache.openjpa.kernel.BrokerImpl.flushSafe(BrokerImpl.java:2011) at org.apache.openjpa.kernel.BrokerImpl.beforeCompletion(BrokerImpl.java:1929) at org.apache.openjpa.kernel.LocalManagedRuntime.commit(LocalManagedRuntime.java:81) at org.apache.openjpa.kernel.BrokerImpl.commit(BrokerImpl.java:1453) at org.apache.openjpa.kernel.DelegatingBroker.commit(DelegatingBroker.java:895) at org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImpl.java:548) ... 8 more Caused by: <openjpa-2.0.0-SNAPSHOT-r422266:774393M fatal general error> org.apache.openjpa.persistence.PersistenceException: ERROR: duplicate key value violates unique constraint "entity_employee_pkey" {prepstmnt 24724157 INSERT INTO ENTITY_EMPLOYEE (id, firstName, lastName, salary) VALUES (?, ?, ?, ?) [params=(int) 256631, (String) 4_First_name_6912, (String) 4_Last_name_6912, (float) 6912.0]} [code=0, state=23505] FailedObject: org.apache.openjpa.persistence.sequence.EntityEmployee@ff5a2b at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DBDictionary.java:4383) at org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(DBDictionary.java:4342) at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:102) at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:72) at org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushAndUpdate(PreparedStatementManagerImpl.java:141) at org.apache.openjpa.jdbc.kernel.BatchingPreparedStatementManagerImpl.flushAndUpdate(BatchingPreparedStatementManagerImpl.java:80) at org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushInternal(PreparedStatementManagerImpl.java:97) at org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flush(PreparedStatementManagerImpl.java:85) at org.apache.openjpa.jdbc.kernel.ConstraintUpdateManager.flush(ConstraintUpdateManager.java:550) at org.apache.openjpa.jdbc.kernel.ConstraintUpdateManager.flush(ConstraintUpdateManager.java:106) at org.apache.openjpa.jdbc.kernel.BatchingConstraintUpdateManager.flush(BatchingConstraintUpdateManager.java:59) at org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdateManager.java:103) at org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdateManager.java:76) at org.apache.openjpa.jdbc.kernel.JDBCStoreManager.flush(JDBCStoreManager.java:672) at org.apache.openjpa.kernel.DelegatingStoreManager.flush(DelegatingStoreManager.java:130) ... 15 more Caused by: org.apache.openjpa.lib.jdbc.ReportingSQLException: ERROR: duplicate key value violates unique constraint "entity_employee_pkey" {prepstmnt 24724157 INSERT INTO ENTITY_EMPLOYEE (id, firstName, lastName, salary) VALUES (?, ?, ?, ?) [params=(int) 256631, (String) 4_First_name_6912, (String) 4_Last_name_6912, (float) 6912.0]} [code=0, state=23505] at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(LoggingConnectionDecorator.java:236) at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.access$700(LoggingConnectionDecorator.java:69) at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator$LoggingConnection$LoggingPreparedStatement.executeUpdate(LoggingConnectionDecorator.java:1080) at org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:286) at org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:286) at org.apache.openjpa.jdbc.kernel.JDBCStoreManager$CancelPreparedStatement.executeUpdate(JDBCStoreManager.java:1579) at org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.executeUpdate(PreparedStatementManagerImpl.java:246) at org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushAndUpdate(PreparedStatementManagerImpl.java:116) ... 25 more Results : Tests in error: testMultiThreadedNativeSequences(org.apache.openjpa.persistence.sequence.TestSequence) Tests run: 1, Failures: 0, Errors: 1, Skipped: 0 [INFO] ------------------------------------------------------------------------ [ERROR] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] There are test failures.
          Hide
          Kevin Sutter added a comment -

          Based on Tim's comments and test case failures, it doesn't look like this JIRA is completely resolved yet. Milosz, please work with Tim to come to a common resolution. Thanks.

          Show
          Kevin Sutter added a comment - Based on Tim's comments and test case failures, it doesn't look like this JIRA is completely resolved yet. Milosz, please work with Tim to come to a common resolution. Thanks.
          Hide
          Milosz Tylenda added a comment -

          Hi Tim, do you run the testcase on a multi-core CPU? What exactly is your change to the testcase?

          I am looking into this and will let know about my findings next week. If I can't help, I will roll my patch back or merge the both patches.

          Show
          Milosz Tylenda added a comment - Hi Tim, do you run the testcase on a multi-core CPU? What exactly is your change to the testcase? I am looking into this and will let know about my findings next week. If I can't help, I will roll my patch back or merge the both patches.
          Hide
          Tim McConnell added a comment -

          Hi Milosz, what I did was change the number of Entities, and number of threads to try to emulate Joe's environment. Try these settings at these lines below. Also, make sure you're using a database that supports native sequences (i.e., nextSequenceQuery in the dictionary – Derby for example, does not).

          00042 private static final int NUMBER_ENTITIES = 30000;
          00264 int iterations = 10;
          00265 int threads = 12;

          Show
          Tim McConnell added a comment - Hi Milosz, what I did was change the number of Entities, and number of threads to try to emulate Joe's environment. Try these settings at these lines below. Also, make sure you're using a database that supports native sequences (i.e., nextSequenceQuery in the dictionary – Derby for example, does not). 00042 private static final int NUMBER_ENTITIES = 30000; 00264 int iterations = 10; 00265 int threads = 12;
          Hide
          Milosz Tylenda added a comment -

          Hi Tim, is it possible that you make the following tests?

          1. Apply the volatile.patch, compile and run the test and see whether it changes anything.
          2. If the above did not help, revert volatile.patch and try to reproduce the error on DB2 or Oracle, so we are sure it is not PostgreSQL fault.

          Show
          Milosz Tylenda added a comment - Hi Tim, is it possible that you make the following tests? 1. Apply the volatile.patch, compile and run the test and see whether it changes anything. 2. If the above did not help, revert volatile.patch and try to reproduce the error on DB2 or Oracle, so we are sure it is not PostgreSQL fault.
          Hide
          Milosz Tylenda added a comment -

          Maybe some volounteer could try reproducing this and if reproduced, try with volatile.patch. If not, I will restore the synchronization around PreparedStatement since I am not able to reproduce the issue with my system (Pentium M).

          Show
          Milosz Tylenda added a comment - Maybe some volounteer could try reproducing this and if reproduced, try with volatile.patch. If not, I will restore the synchronization around PreparedStatement since I am not able to reproduce the issue with my system (Pentium M).
          Hide
          Tim McConnell added a comment -

          Hi Milosz, I'll apply your volatile patch later today and let you know what happens. Note though that the failure can I can reproduce the problem on DB2, Oracle, and PostgreSQL – it's certainly not isolated to PostgreSQL. Talk later....

          Show
          Tim McConnell added a comment - Hi Milosz, I'll apply your volatile patch later today and let you know what happens. Note though that the failure can I can reproduce the problem on DB2, Oracle, and PostgreSQL – it's certainly not isolated to PostgreSQL. Talk later....
          Hide
          Tim McConnell added a comment -

          Hi Milosz, here is what I've done over the past couple days. I have two machines – Windows XP, Windows 2003 – each with MySQL (8.3 on one machine, 8.4 on the other machine), DB2 (8.2 and 9.5), and Oracle (10g on both) installed on them. To reproduce the original problem I use 25000 entities, 6 iterations, and 8 threads in the TestSequence test program. With these settings I can easily reproduce the failure on both machines on all three databases. Here are the results of the various patches attached to this JIRA. Note that I raised the number entities to 40000 to stress each patch (but still used 6 iterations and 8 threads):

          1. Patch OPENJPA-466 (attached by you with the change to AbstractJDBCSeq.java): Failures on all three databases on both machines. Note that the failures manifest themselves differently on each database. For example the failures on PostgreSQL show up as

          ERROR: duplicate key value violates unique constraint

          on Oracle as: ORA-00001: unique constraint (SYSTEM.SYS_C008294) violated

          and finally on DB2 as: org.apache.openjpa.persistence.EntityExistsException: DB2 SQL error: SQLCODE: -803, SQLSTATE: 23505

          2. Patch volite.patch only (i.e., no other patches used): Failures on all three databases on both machines

          3. Patch OPENJPA-466-SYNCHRONIZED.patch only (which is a modified version of my original patch but is what was originally committed to trunk): Success on all three databases on both machines.

          So here are my recommendations:

          The OPENJPA-466-SYNCHRONIZED patch works in all scenarios that I've tested it in. More importantly, it fixed the original problem reported by the user, and has no adverse impact on performance. Please note as well that that user environment was an extremely large, high-volume, multi-threaded, multi-core installation. Not meaning to be overly brusque – but it simply works in every scenario that it's been used it. Until or unless someone (other than me as I don't intend to spend any more time on this JIRA) can provide a scenario where it does not work it should be committed to trunk and retrofitted to all previous versions of OpenJPA, and any previously-committed patches should be reverted. Thanks much

          Show
          Tim McConnell added a comment - Hi Milosz, here is what I've done over the past couple days. I have two machines – Windows XP, Windows 2003 – each with MySQL (8.3 on one machine, 8.4 on the other machine), DB2 (8.2 and 9.5), and Oracle (10g on both) installed on them. To reproduce the original problem I use 25000 entities, 6 iterations, and 8 threads in the TestSequence test program. With these settings I can easily reproduce the failure on both machines on all three databases. Here are the results of the various patches attached to this JIRA. Note that I raised the number entities to 40000 to stress each patch (but still used 6 iterations and 8 threads): 1. Patch OPENJPA-466 (attached by you with the change to AbstractJDBCSeq.java): Failures on all three databases on both machines. Note that the failures manifest themselves differently on each database. For example the failures on PostgreSQL show up as ERROR: duplicate key value violates unique constraint on Oracle as: ORA-00001: unique constraint (SYSTEM.SYS_C008294) violated and finally on DB2 as: org.apache.openjpa.persistence.EntityExistsException: DB2 SQL error: SQLCODE: -803, SQLSTATE: 23505 2. Patch volite.patch only (i.e., no other patches used): Failures on all three databases on both machines 3. Patch OPENJPA-466 -SYNCHRONIZED.patch only (which is a modified version of my original patch but is what was originally committed to trunk): Success on all three databases on both machines. So here are my recommendations: The OPENJPA-466 -SYNCHRONIZED patch works in all scenarios that I've tested it in. More importantly, it fixed the original problem reported by the user, and has no adverse impact on performance. Please note as well that that user environment was an extremely large, high-volume, multi-threaded, multi-core installation. Not meaning to be overly brusque – but it simply works in every scenario that it's been used it. Until or unless someone (other than me as I don't intend to spend any more time on this JIRA) can provide a scenario where it does not work it should be committed to trunk and retrofitted to all previous versions of OpenJPA, and any previously-committed patches should be reverted. Thanks much
          Hide
          Donald Woods added a comment -

          Tim, not sure why you marked this as Resolved when your latest patch hasn't been committed yet.....

          Show
          Donald Woods added a comment - Tim, not sure why you marked this as Resolved when your latest patch hasn't been committed yet.....
          Hide
          Michael Dick added a comment -

          I think we're up to date in trunk and 1.3.x now. Milosz, Frank or Tim, if there are additional changes that need to be made please re-open the issue.

          Thanks for the patches and the time you spend debugging this problem Tim.

          Show
          Michael Dick added a comment - I think we're up to date in trunk and 1.3.x now. Milosz, Frank or Tim, if there are additional changes that need to be made please re-open the issue. Thanks for the patches and the time you spend debugging this problem Tim.
          Hide
          Milosz Tylenda added a comment -

          The volatile.patch was meant to be applied to what was on the trunk without any reverts so it is not a surprise that "2. Patch volite.patch only (i.e., no other patches used): Failures on all three databases on both machines".

          I agree with applying the latest patch and resolving this. This is better that what we had before and it seems there is not enough interest in testing the patches.

          I do state however we just hid a multi-threading bug that can bite us later.

          Show
          Milosz Tylenda added a comment - The volatile.patch was meant to be applied to what was on the trunk without any reverts so it is not a surprise that "2. Patch volite.patch only (i.e., no other patches used): Failures on all three databases on both machines". I agree with applying the latest patch and resolving this. This is better that what we had before and it seems there is not enough interest in testing the patches. I do state however we just hid a multi-threading bug that can bite us later.
          Hide
          B.J. Reed added a comment -

          Attached 2 patches to port necessary code back to 1.0.x and 1.2.x.

          Show
          B.J. Reed added a comment - Attached 2 patches to port necessary code back to 1.0.x and 1.2.x.
          Hide
          Donald Woods added a comment -

          add missing Fix versions of 1..0.4 and 1.2.2

          Show
          Donald Woods added a comment - add missing Fix versions of 1..0.4 and 1.2.2
          Hide
          Joe Pullen added a comment -

          Today came back to this issue get the patch and was surprised that after the first fix there was more discussion. Just wanted to say from my view the original patch from Tim did the job perfectly.

          As Tim stated I can tell you that the original patch was def needed and helped solve a major issue with an application running on 4 T2000 Solaris machines (each with 64 cores, yes 256 cores !). This J2EE app has processed 10 millions of business transactions with throughput in peak of 100,000s business transactions (equals to 250,000 JTA and over a million SQL statements) per hour without a SINGLE duplicate primary key error related to sequences.

          Show
          Joe Pullen added a comment - Today came back to this issue get the patch and was surprised that after the first fix there was more discussion. Just wanted to say from my view the original patch from Tim did the job perfectly. As Tim stated I can tell you that the original patch was def needed and helped solve a major issue with an application running on 4 T2000 Solaris machines (each with 64 cores, yes 256 cores !). This J2EE app has processed 10 millions of business transactions with throughput in peak of 100,000s business transactions (equals to 250,000 JTA and over a million SQL statements) per hour without a SINGLE duplicate primary key error related to sequences.
          Hide
          Pinaki Poddar added a comment -

          The issue of sequence value creation under multithreaded environment is still relevant.
          I ran a very simple tests that do the following
          1. A entity with @GeneratedValue primary key
          2. 4 threads that persist() entities in separate persistence contexts

          The test breaks with OPENJPA_SEQUENCE_TABLE being populated with duplicate key – looks like a concurrency issue.

          Show
          Pinaki Poddar added a comment - The issue of sequence value creation under multithreaded environment is still relevant. I ran a very simple tests that do the following 1. A entity with @GeneratedValue primary key 2. 4 threads that persist() entities in separate persistence contexts The test breaks with OPENJPA_SEQUENCE_TABLE being populated with duplicate key – looks like a concurrency issue.
          Hide
          Pinaki Poddar added a comment -

          The test was run onm MYSQL 5.3.

          Show
          Pinaki Poddar added a comment - The test was run onm MYSQL 5.3.
          Hide
          Pinaki Poddar added a comment -

          A test and a entity to demonstrate the concurrency issue in sequence value generation.

          Show
          Pinaki Poddar added a comment - A test and a entity to demonstrate the concurrency issue in sequence value generation.
          Hide
          Milosz Tylenda added a comment -

          I confirm the issue although I had to modify the test to call em.persist in a loop and be a bit patient.

          We are probably failing during OPENJPA_SEQUENCE_TABLE initialization, not during generating numbers from it.

          Show
          Milosz Tylenda added a comment - I confirm the issue although I had to modify the test to call em.persist in a loop and be a bit patient. We are probably failing during OPENJPA_SEQUENCE_TABLE initialization, not during generating numbers from it.
          Hide
          Milosz Tylenda added a comment -

          I will create a new issue for the sequence table problem as it is loosely coupled with the original issue. I guess the sequence table failure is the known problem where multiple threads access an uninitailized emf, it has been reported on the users mailing list.

          Show
          Milosz Tylenda added a comment - I will create a new issue for the sequence table problem as it is loosely coupled with the original issue. I guess the sequence table failure is the known problem where multiple threads access an uninitailized emf, it has been reported on the users mailing list.

            People

            • Assignee:
              Milosz Tylenda
              Reporter:
              Frank Le
            • Votes:
              5 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development