Derby
  1. Derby
  2. DERBY-1327

Identity column can be created with wrong and very large start with value with "J2RE 1.5.0 IBM Windows 32 build pwi32dev-20060412 (SR2)" with JIT on

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 10.1.3.1, 10.2.1.6
    • Fix Version/s: 10.1.3.1, 10.2.1.6
    • Component/s: SQL
    • Labels:
      None

      Description

      Using the following JRE with JIT on an identity column may be created with a wrong and very large START WITH value. When the problem occurs it affects not only the table being created, but also other tables that were created in previous transactions.

      For example attempting to create 1000 tables with identity columns the 126th table creation changes the start with value in sys.syscolumns to 41628850257395713 for ALL 125 tables. Attempts to insert into any of the tables cause
      "SQL Exception: A truncation error was encountered trying to
      shrink ... to length 12."

      This program will create up to 1000 tables until the problem
      occurs
      Note:

      • The problem does not occur with -Xnojit (JIT OFF)
      • The problem, when it occurs, changes not only the table being
        created but all previous tables created. See output below.
        Every thing was fine up until mytable126 and then all the
        tables got changed to start with 41628850257395713
      • Problem occurs with autocommit on/off.
      • The problem occurs after the create table but before the
        commit.
      • If the non-identity columns are removed the problem does not
        reproduce.

      import java.sql.DatabaseMetaData;
      import java.sql.ResultSet;
      import java.sql.SQLException;
      import java.sql.Statement;
      import java.sql.DriverManager;

      public class BadStartWith
      {

      public static void main (String args [])throws Exception

      { testBadStartWith(); }

      /**

      • After some number of table creations with JIT turned on, the START WITH value
      • for the table being created and all the ones already created gets mysteriously
      • changed with pwi32dev-20060412 (SR2)
      • @throws Exception
        */
        public static void testBadStartWith() throws Exception
        {

      Class.forName("org.apache.derby.jdbc.EmbeddedDriver").newInstance();
      Connection conn = DriverManager.getConnection("jdbc:derby:wombat;create=true");
      conn.setAutoCommit(false);
      Statement stmt = null;

      DatabaseMetaData md = conn.getMetaData() ;
      System.out.println(md.getDatabaseProductVersion());
      System.out.println(md.getDatabaseProductName());
      System.out.println(md.getDriverName());
      dropAllAppTables(conn);
      System.out.println("Create tables until we get a wrong Start with value");
      stmt = conn.createStatement();

      // numBadStartWith will be changed if any columns get a bad start with value.
      int numBadStartWith = 0;

      try {
      // create 1000 tables. Break out if we get a table that has a bad
      // start with value.
      for (int i = 0; (i < 1000) && (numBadStartWith == 0); i++)

      { String tableName = "APP.MYTABLE" + i; String createTableSQL = "CREATE TABLE " + tableName + " (ROLEID INTEGER NOT NULL GENERATED ALWAYS AS IDENTITY (START WITH 2, INCREMENT BY 1), INSTANCEID INTEGER, STATUS INTEGER, LOGICAL_STATE INTEGER, LSTATE_TSTAMP TIMESTAMP, UPDT_TSTAMP TIMESTAMP, TSTAMP TIMESTAMP, CLALEVEL1_CLALEVEL2_CLALEVEL2ID VARCHAR(255), CLALEVEL1_CLALEVEL2_CLALEVEL3_CLALEVEL3ID VARCHAR(255))"; stmt.executeUpdate(createTableSQL); System.out.println(createTableSQL); System.out.println("Check before commit"); numBadStartWith = checkBadStartWithCols(conn,2); conn.commit(); System.out.println("Check after commit"); numBadStartWith = checkBadStartWithCols(conn,2); if (numBadStartWith > 0) break; }

      } catch (SQLException se)

      { se.printStackTrace(); }

      if (numBadStartWith == 0)
      System.out.println("PASS: All 1000 tables created without problems");
      stmt.close();
      conn.rollback();
      conn.close();
      }

      /**

      • Check that all tables in App do not have a an autoincrementstart value
      • greater tan maxautoincrementstart
      • @param conn
      • @param maxautoincrementstart Maximum expected autoincrementstart value
      • @return number of columns with bad autoincrementstart value
        */
        private static int checkBadStartWithCols(Connection conn, int
        maxautoincrementstart) throws Exception
        {
        Statement stmt = conn.createStatement();
        ResultSet rs =stmt.executeQuery("select count(autoincrementstart) from sys.syscolumns c, sys.systables t, sys.sysschemas s WHERE t.schemaid = s.schemaid and s.schemaname = 'APP' and autoincrementstart > " + maxautoincrementstart);

      rs.next();
      int numBadStartWith = rs.getInt(1);
      System.out.println(numBadStartWith + " columns have bad START WITH VALUE");
      rs.close();

      if (numBadStartWith > 0)
      {
      rs =stmt.executeQuery("select tablename, columnname, autoincrementstart from sys.syscolumns c, sys.systables t, sys.sysschemas s WHERE t.schemaid = s.schemaid and s.schemaname = 'APP' and autoincrementstart > 2 ORDER BY tablename");
      while (rs.next())

      { System.out.println("Unexpected start value: " + rs.getLong(3) + " on column " + rs.getString(1) + "(" + rs.getString(2) + ")"); }

      }
      return numBadStartWith;
      }

      /**

      • Drop all tables in schema APP
      • @param conn
      • @throws SQLException
        */
        private static void dropAllAppTables(Connection conn) throws SQLException
        {
        Statement stmt1 = conn.createStatement();
        Statement stmt2 = conn.createStatement();
        System.out.println("Drop all tables in APP schema");
        ResultSet rs = stmt1.executeQuery("SELECT tablename from sys.systables t, sys.sysschemas s where t.schemaid = s.schemaid and s.schemaname = 'APP'");

      while (rs.next())
      {
      String tableName = rs.getString(1);

      try

      { stmt2.executeUpdate("DROP TABLE " + tableName); }

      catch (SQLException se)

      { System.out.println("Error dropping table:" + tableName); se.printStackTrace(); continue; }

      }
      }

      }

      Relevant output:
      $java BadStartWith
      10.2.0.0 alpha
      Apache Derby
      Apache Derby Embedded JDBC Driver
      Drop all tables in APP schema
      Create tables until we get a wrong Start with value
      CREATE TABLE APP.MYTABLE0 (ROLEID INTEGER NOT NULL GENERATED ALWAYS AS IDENTITY
      (START WITH 2, INCREMENT BY 1), INSTANCEID INTEGER, STATUS INTEGER, LOGICAL_STA
      TE INTEGER, LSTATE_TSTAMP TIMESTAMP, UPDT_TSTAMP TIMESTAMP, TSTAMP TIMESTAMP, C
      LALEVEL1_CLALEVEL2_CLALEVEL2ID VARCHAR(255), CLALEVEL1_CLALEVEL2_CLALEVEL3_CLAL
      EVEL3ID VARCHAR(255))
      Check before commit
      0 columns have bad START WITH VALUE
      Check after commit
      0 columns have bad START WITH VALUE
      [snip MYTABLE1 ... MYTABLE124]

      CREATE TABLE APP.MYTABLE125 (ROLEID INTEGER NOT NULL GENERATED ALWAYS AS IDENTITY (START WITH 2, INCREMENT BY 1), INSTANCEID INTEGER, STATUS INTEGER, LOGICAL_STATE INTEGER, LSTATE_TSTAMP TIMESTAMP,
      UPDT_TSTAMP TIMESTAMP, TSTAMP TIMESTAMP, CLALEVEL1_CLALEVEL2_CLALEVEL2ID VARCHAR(255), CLALEVEL1_CLALEVEL2_CLALEVEL3_CLALEVEL3ID VARCHAR(255))
      Check before commit
      0 columns have bad START WITH VALUE
      Check after commit
      0 columns have bad START WITH VALUE
      CREATE TABLE APP.MYTABLE126 (ROLEID INTEGER NOT NULL GENERATED ALWAYS AS IDENTITY (START WITH 2, INCREMENT BY 1), INSTANCEID INTEGER, STATUS INTEGER, LOGICAL_STATE INTEGER, LSTATE_TSTAMP TIMESTAMP,
      UPDT_TSTAMP TIMESTAMP, TSTAMP TIMESTAMP, CLALEVEL1_CLALEVEL2_CLALEVEL2ID VARCHAR(255), CLALEVEL1_CLALEVEL2_CLALEVEL3_CLALEVEL3ID VARCHAR(255))
      Check before commit
      127 columns have bad START WITH VALUE
      Unexpected start value: 41628850257395713 on column MYTABLE0(ROLEID)
      Unexpected start value: 41628850257395713 on column MYTABLE1(ROLEID)
      Unexpected start value: 41628850257395713 on column MYTABLE10(ROLEID)
      Unexpected start value: 41628850257395713 on column MYTABLE100(ROLEID)
      Unexpected start value: 41628850257395713 on column MYTABLE101(ROLEID)
      Unexpected start value: 41628850257395713 on column MYTABLE102(ROLEID)
      Unexpected start value: 41628850257395713 on column MYTABLE103(ROLEID)
      Unexpected start value: 41628850257395713 on column MYTABLE104(ROLEID)
      Unexpected start value: 41628850257395713 on column MYTABLE105(ROLEID)
      Unexpected start value: 41628850257395713 on column MYTABLE106(ROLEID)
      Unexpected start value: 41628850257395713 on column MYTABLE107(ROLEID)
      Unexpected start value: 41628850257395713 on column MYTABLE108(ROLEID)
      Unexpected start value: 41628850257395713 on column MYTABLE109(ROLEID)
      Unexpected start value: 41628850257395713 on column MYTABLE11(ROLEID)
      Unexpected start value: 41628850257395713 on column MYTABLE110(ROLEID)
      [snip the rest of the tables have unexpected START WITH value too]

      1. Derby1327WrongStartKeyPatch1CodelineTrunk.txt
        483 kB
        Mamta A. Satoor
      2. Derby1327WrongStartKeyPatch1SvnDiff101.txt
        474 kB
        Mamta A. Satoor
      3. Derby1327WrongStartKeyPatch1SvnStat101.txt
        2 kB
        Mamta A. Satoor

        Activity

        Hide
        Kathey Marsden added a comment -

        I am trying to narrrow down what java calls may be causing this issue.
        Because it only reproduces with JIT on I can't really debug with the debugger.
        How is it that the creation of one table could affect the autoincrementstart value of all the tables that have already been created and committed?

        I am not able to post this JRE unfortunately but am hoping someone can provide some pointers on how one create table could have such a far ranging impact on tables already created.

        I tried putting some println's in org.apache.derby.impl.sql.execute.ColumnInfo but never see autoincstart being greater than 2. Other references to this value seem unlikely to cause such widespread damage to other tables. I'd appreciate any clues or thoughts on how to track this down.

        Thanks

        Kathey

        Show
        Kathey Marsden added a comment - I am trying to narrrow down what java calls may be causing this issue. Because it only reproduces with JIT on I can't really debug with the debugger. How is it that the creation of one table could affect the autoincrementstart value of all the tables that have already been created and committed? I am not able to post this JRE unfortunately but am hoping someone can provide some pointers on how one create table could have such a far ranging impact on tables already created. I tried putting some println's in org.apache.derby.impl.sql.execute.ColumnInfo but never see autoincstart being greater than 2. Other references to this value seem unlikely to cause such widespread damage to other tables. I'd appreciate any clues or thoughts on how to track this down. Thanks Kathey
        Hide
        Kathey Marsden added a comment -

        I found some good JIT diagnostic info for this JVM at:
        http://download.boulder.ibm.com/ibmdl/pub/software/dw/jdk/diagnosis/diag50.pdf

        To get jit to kick in immediately so it fails on the first create table and get a dump of the methods compiled I:

        java -Xjit:count=0,disableInlining,optLevel=noOpt,verbose,vlog=BadStartWith.vlog BadStartWith

        I did a binary search to find the offending entry narrowed down to line 5531
        java -Xjit:count=0,disableInlining,optLevel=noOpt,limitFile=(BadStartWith.vlog,5531,5531) BadStartWith

        This is the offending compilation:

        + (no-opt) org/apache/derby/impl/sql/execute/CreateTableConstantAction.executeConstantAction(Lorg/apache/derby/iapi/sql/Activation;)V @ 0x229499B0-0x2294A9FA

        So the workaround to exclude compilation of this method is:

        java -Xjit:exclude=

        {org/apache/derby/impl/sql/execute/CreateTableConstantAction.executeConstantAction\(Lorg/apache/derby/iapi/sql/Activation\;\)V}

        BadStartWith

        Show
        Kathey Marsden added a comment - I found some good JIT diagnostic info for this JVM at: http://download.boulder.ibm.com/ibmdl/pub/software/dw/jdk/diagnosis/diag50.pdf To get jit to kick in immediately so it fails on the first create table and get a dump of the methods compiled I: java -Xjit:count=0,disableInlining,optLevel=noOpt,verbose,vlog=BadStartWith.vlog BadStartWith I did a binary search to find the offending entry narrowed down to line 5531 java -Xjit:count=0,disableInlining,optLevel=noOpt,limitFile=(BadStartWith.vlog,5531,5531) BadStartWith This is the offending compilation: + (no-opt) org/apache/derby/impl/sql/execute/CreateTableConstantAction.executeConstantAction(Lorg/apache/derby/iapi/sql/Activation;)V @ 0x229499B0-0x2294A9FA So the workaround to exclude compilation of this method is: java -Xjit:exclude= {org/apache/derby/impl/sql/execute/CreateTableConstantAction.executeConstantAction\(Lorg/apache/derby/iapi/sql/Activation\;\)V} BadStartWith
        Hide
        Kathey Marsden added a comment -

        The problem seems to be in the compilation of this constructor call.
        The parameter for autoincStart gets compiled incorrectly.
        This is a long value and even hard coded as 0L it will have a bad value maybe an address, since the 0L value is much less than the variable. I still have not been able to isolate completely from Derby.

        Constructor call code in CreateTAbleConstantAction.executeConstantAction()

        columnDescriptor = new ColumnDescriptor(
        columnInfo[ix].name,
        index++,
        columnInfo[ix].dataType,
        columnInfo[ix].defaultValue,
        columnInfo[ix].defaultInfo,
        td,
        defaultUUID,
        // below is the problem parameter.
        // Occurs also with 0L hard coded.
        // The value when printed is ok.
        columnInfo[ix].autoincStart,
        columnInfo[ix].autoincInc,
        columnInfo[ix].autoincInc != 0,
        columnInfo[ix].autoinc_create_or_modify_Start_Increment
        );

        Constructor definition:

        public ColumnDescriptor(String columnName, int columnPosition,
        DataTypeDescriptor columnType, DataValueDescriptor columnDefault,
        DefaultInfo columnDefaultInfo,
        TableDescriptor table,
        UUID defaultUUID, long autoincStart,
        long autoincInc, boolean
        autoinc,
        long userChangedWhat)

        Show
        Kathey Marsden added a comment - The problem seems to be in the compilation of this constructor call. The parameter for autoincStart gets compiled incorrectly. This is a long value and even hard coded as 0L it will have a bad value maybe an address, since the 0L value is much less than the variable. I still have not been able to isolate completely from Derby. Constructor call code in CreateTAbleConstantAction.executeConstantAction() columnDescriptor = new ColumnDescriptor( columnInfo [ix] .name, index++, columnInfo [ix] .dataType, columnInfo [ix] .defaultValue, columnInfo [ix] .defaultInfo, td, defaultUUID, // below is the problem parameter. // Occurs also with 0L hard coded. // The value when printed is ok. columnInfo [ix] .autoincStart, columnInfo [ix] .autoincInc, columnInfo [ix] .autoincInc != 0, columnInfo [ix] .autoinc_create_or_modify_Start_Increment ); Constructor definition: public ColumnDescriptor(String columnName, int columnPosition, DataTypeDescriptor columnType, DataValueDescriptor columnDefault, DefaultInfo columnDefaultInfo, TableDescriptor table, UUID defaultUUID, long autoincStart, long autoincInc, boolean autoinc, long userChangedWhat)
        Hide
        Kathey Marsden added a comment -

        The information that I got on this issue is that the problem is triggered when a method with a long parameter list (>10) is called, but before the JIT compiler has encountered the same number of temporary variables.

        A possible Derby Workaround would be to rework this code so the constructor to have ten or less parameters

        Show
        Kathey Marsden added a comment - The information that I got on this issue is that the problem is triggered when a method with a long parameter list (>10) is called, but before the JIT compiler has encountered the same number of temporary variables. A possible Derby Workaround would be to rework this code so the constructor to have ten or less parameters
        Hide
        Mamta A. Satoor added a comment -

        The fix for this issue would be to reduce the number of parameters required by the constructor to <=10 in org.apache.derby.iapi.sql.dictionary.ColumnDescriptor class. While researching into this, I found that all the 3 constructors in the class have a parameter named autoinc and it is defined as a boolean. This parameter is always equal to (parameter named autoincInc != 0). In my patch (Derby1327WrongStartKeyPatch1CodelineTrunk.txt) which is attached to this JIRA, I have removed the autoinc parameter and inside the constructors, I use (parameter named autoincInc != 0) instead of relying on autoinc. This cleans up the constructor parameter passing for all the 3 constructors and also brings down the number of parameters to <=10. The test program from the JIRA entry runs fine with this change and I have created a new test JitTest.java based on that test program. Hopefully this test can be a place holder for any future JIT issues. I also ran the test suites and there were no new failures.

        If the changes looks good, can a committer please commit it? As a cleanup, I also removed the import of org.apache.derby.iapi.sql.dictionary.ColumnDescriptor from some classes which didn't really use ColumnDescriptor.

        svn stat
        M java\engine\org\apache\derby\impl\sql\compile\FromSubquery.java
        M java\engine\org\apache\derby\impl\sql\compile\CallStatementNode.java
        M java\engine\org\apache\derby\impl\sql\compile\UpdateNode.java
        M java\engine\org\apache\derby\impl\sql\compile\FromList.java
        M java\engine\org\apache\derby\impl\sql\compile\InsertNode.java
        M java\engine\org\apache\derby\impl\sql\compile\SubqueryList.java
        M java\engine\org\apache\derby\impl\sql\compile\ResultColumnList.java
        M java\engine\org\apache\derby\impl\sql\execute\WriteCursorConstantAction.java
        M java\engine\org\apache\derby\impl\sql\execute\GenericResultSetFactory.java
        M java\engine\org\apache\derby\impl\sql\execute\DropSchemaConstantAction.java
        M java\engine\org\apache\derby\impl\sql\execute\CreateTableConstantAction.java
        M java\engine\org\apache\derby\impl\sql\execute\CreateViewConstantAction.java
        M java\engine\org\apache\derby\impl\sql\execute\IndexConstantAction.java
        M java\engine\org\apache\derby\impl\sql\execute\DropConstraintConstantAction.java
        M java\engine\org\apache\derby\impl\sql\execute\SetConstraintsConstantAction.java
        M java\engine\org\apache\derby\impl\sql\execute\InsertConstantAction.java
        M java\engine\org\apache\derby\impl\sql\execute\DropViewConstantAction.java
        M java\engine\org\apache\derby\impl\sql\execute\ConstraintConstantAction.java
        M java\engine\org\apache\derby\impl\sql\execute\DropIndexConstantAction.java
        M java\engine\org\apache\derby\impl\sql\execute\AlterTableConstantAction.java
        M java\engine\org\apache\derby\impl\sql\catalog\SYSCOLUMNSRowFactory.java
        M java\engine\org\apache\derby\impl\sql\catalog\DataDictionaryImpl.java
        M java\engine\org\apache\derby\impl\sql\catalog\SYSFILESRowFactory.java
        M java\engine\org\apache\derby\iapi\sql\conn\LanguageConnectionContext.java
        M java\engine\org\apache\derby\iapi\sql\dictionary\ColumnDescriptor.java
        A java\testing\org\apache\derbyTesting\functionTests\tests\lang\JitTest.java
        A java\testing\org\apache\derbyTesting\functionTests\master\JitTest.out
        M java\testing\org\apache\derbyTesting\functionTests\suites\derbylang.runall

        Show
        Mamta A. Satoor added a comment - The fix for this issue would be to reduce the number of parameters required by the constructor to <=10 in org.apache.derby.iapi.sql.dictionary.ColumnDescriptor class. While researching into this, I found that all the 3 constructors in the class have a parameter named autoinc and it is defined as a boolean. This parameter is always equal to (parameter named autoincInc != 0). In my patch (Derby1327WrongStartKeyPatch1CodelineTrunk.txt) which is attached to this JIRA, I have removed the autoinc parameter and inside the constructors, I use (parameter named autoincInc != 0) instead of relying on autoinc. This cleans up the constructor parameter passing for all the 3 constructors and also brings down the number of parameters to <=10. The test program from the JIRA entry runs fine with this change and I have created a new test JitTest.java based on that test program. Hopefully this test can be a place holder for any future JIT issues. I also ran the test suites and there were no new failures. If the changes looks good, can a committer please commit it? As a cleanup, I also removed the import of org.apache.derby.iapi.sql.dictionary.ColumnDescriptor from some classes which didn't really use ColumnDescriptor. svn stat M java\engine\org\apache\derby\impl\sql\compile\FromSubquery.java M java\engine\org\apache\derby\impl\sql\compile\CallStatementNode.java M java\engine\org\apache\derby\impl\sql\compile\UpdateNode.java M java\engine\org\apache\derby\impl\sql\compile\FromList.java M java\engine\org\apache\derby\impl\sql\compile\InsertNode.java M java\engine\org\apache\derby\impl\sql\compile\SubqueryList.java M java\engine\org\apache\derby\impl\sql\compile\ResultColumnList.java M java\engine\org\apache\derby\impl\sql\execute\WriteCursorConstantAction.java M java\engine\org\apache\derby\impl\sql\execute\GenericResultSetFactory.java M java\engine\org\apache\derby\impl\sql\execute\DropSchemaConstantAction.java M java\engine\org\apache\derby\impl\sql\execute\CreateTableConstantAction.java M java\engine\org\apache\derby\impl\sql\execute\CreateViewConstantAction.java M java\engine\org\apache\derby\impl\sql\execute\IndexConstantAction.java M java\engine\org\apache\derby\impl\sql\execute\DropConstraintConstantAction.java M java\engine\org\apache\derby\impl\sql\execute\SetConstraintsConstantAction.java M java\engine\org\apache\derby\impl\sql\execute\InsertConstantAction.java M java\engine\org\apache\derby\impl\sql\execute\DropViewConstantAction.java M java\engine\org\apache\derby\impl\sql\execute\ConstraintConstantAction.java M java\engine\org\apache\derby\impl\sql\execute\DropIndexConstantAction.java M java\engine\org\apache\derby\impl\sql\execute\AlterTableConstantAction.java M java\engine\org\apache\derby\impl\sql\catalog\SYSCOLUMNSRowFactory.java M java\engine\org\apache\derby\impl\sql\catalog\DataDictionaryImpl.java M java\engine\org\apache\derby\impl\sql\catalog\SYSFILESRowFactory.java M java\engine\org\apache\derby\iapi\sql\conn\LanguageConnectionContext.java M java\engine\org\apache\derby\iapi\sql\dictionary\ColumnDescriptor.java A java\testing\org\apache\derbyTesting\functionTests\tests\lang\JitTest.java A java\testing\org\apache\derbyTesting\functionTests\master\JitTest.out M java\testing\org\apache\derbyTesting\functionTests\suites\derbylang.runall
        Hide
        Mamta A. Satoor added a comment -

        BTW, the patch, Derby1327WrongStartKeyPatch1CodelineTrunk.txt is for the main codeline. If it looks good and there are no comments on it, I can submit similar patch for 10.1 codeline as well.

        Show
        Mamta A. Satoor added a comment - BTW, the patch, Derby1327WrongStartKeyPatch1CodelineTrunk.txt is for the main codeline. If it looks good and there are no comments on it, I can submit similar patch for 10.1 codeline as well.
        Hide
        Satheesh Bandaram added a comment -

        I was reviewing this change and I noticed we are moving this:

        • /* NOTE: We use the autoincColumn variable in order to work around
        • * a 1.3.0 HotSpot bug. (#4361550)
        • */
        • boolean autoincColumn = (autoincInc != 0);

        Does anyone know if this "HotSpot" bug is still an issue and any issues caused by removing this code?

        Noticed this code has been there since 10.0... so likely came from before code contribution.

        Show
        Satheesh Bandaram added a comment - I was reviewing this change and I noticed we are moving this: /* NOTE: We use the autoincColumn variable in order to work around * a 1.3.0 HotSpot bug. (#4361550) */ boolean autoincColumn = (autoincInc != 0); Does anyone know if this "HotSpot" bug is still an issue and any issues caused by removing this code? Noticed this code has been there since 10.0... so likely came from before code contribution.
        Hide
        Mamta A. Satoor added a comment -

        While working on this, I tried looking up the number 4361550 on Sun's website and got only one hit(http://sunsolve.sun.com/search/document.do?assetkey=1-1-4361550-1), which I could not access because I don't have SunSolve technical support id.

        The code that I am removing was first added in Cloudscape in August 2000 for 1.3.0 HotSpot bug which we don't support any more(I think). So, I am hoping my code change will not have any regression.

        Show
        Mamta A. Satoor added a comment - While working on this, I tried looking up the number 4361550 on Sun's website and got only one hit( http://sunsolve.sun.com/search/document.do?assetkey=1-1-4361550-1 ), which I could not access because I don't have SunSolve technical support id. The code that I am removing was first added in Cloudscape in August 2000 for 1.3.0 HotSpot bug which we don't support any more(I think). So, I am hoping my code change will not have any regression.
        Hide
        Kathey Marsden added a comment -

        Hopefully it is safe to assume that any hot spot bug from 1.3.0, August 2000 has been rectified in the currently supported JDK 1.3.1.
        I think running the new test with Sun JDK 1.3.1 would be good verification. Beyojnd that unless someone has access and wants to look this up I don't think we need to keep that code.

        Kind of ironic that cleaning up a workaround to fix one JIT bug, fixed another JIT bug.

        Kathey

        Show
        Kathey Marsden added a comment - Hopefully it is safe to assume that any hot spot bug from 1.3.0, August 2000 has been rectified in the currently supported JDK 1.3.1. I think running the new test with Sun JDK 1.3.1 would be good verification. Beyojnd that unless someone has access and wants to look this up I don't think we need to keep that code. Kind of ironic that cleaning up a workaround to fix one JIT bug, fixed another JIT bug. Kathey
        Hide
        Mamta A. Satoor added a comment -

        I ran the new test JitTest.java with Sun JDK 1.3.1 and it ran fine. I have also started derbyall run with Sun JDK 1.3.1 and so far no failures. This is all on main codeline.

        Show
        Mamta A. Satoor added a comment - I ran the new test JitTest.java with Sun JDK 1.3.1 and it ran fine. I have also started derbyall run with Sun JDK 1.3.1 and so far no failures. This is all on main codeline.
        Hide
        Mamta A. Satoor added a comment -

        Sun JDK 1.3.1 run on derbyall finished on my codeline with no new failures. So, if there are no further comments, can a committer please commit this patch?

        Show
        Mamta A. Satoor added a comment - Sun JDK 1.3.1 run on derbyall finished on my codeline with no new failures. So, if there are no further comments, can a committer please commit this patch?
        Hide
        Kathey Marsden added a comment -

        On the test JitTest.java I am concerned with the time that it takes. It took 6 minutes.
        I think creating 200 tables would be more than sufficient in this case and bring it down to 2. Ok if I make that small edit before checking in?

        Show
        Kathey Marsden added a comment - On the test JitTest.java I am concerned with the time that it takes. It took 6 minutes. I think creating 200 tables would be more than sufficient in this case and bring it down to 2. Ok if I make that small edit before checking in?
        Hide
        Mamta A. Satoor added a comment -

        Sure, Kathey, go ahead and make that change.

        Show
        Mamta A. Satoor added a comment - Sure, Kathey, go ahead and make that change.
        Hide
        Kathey Marsden added a comment -

        Committed this to trunk. I reduced some superfluous before and after commit checking and reduced the test output to speed things up so JitTest runs in under 1:30.

        Date: Fri Jun 9 12:23:47 2006
        New Revision: 413129

        URL: http://svn.apache.org/viewvc?rev=413129&view=rev

        Show
        Kathey Marsden added a comment - Committed this to trunk. I reduced some superfluous before and after commit checking and reduced the test output to speed things up so JitTest runs in under 1:30. Date: Fri Jun 9 12:23:47 2006 New Revision: 413129 URL: http://svn.apache.org/viewvc?rev=413129&view=rev
        Hide
        Mamta A. Satoor added a comment -

        I am attaching the patch for 10.1 codeline after doing a merge from main and hand fixing some of the conflicts. (The derbyall test suite ran fine with jdk13)
        svn merge -r 413128:413129 https://svn.apache.org/repos/asf/db/derby/code/trunk/

        One thing to note though is that in 10.1 codeline, the number of parameters to ColumnDescriptor constructors are 10 and not >10. So, from the bug diagnosis as mentioned in this JIRA entry, the bug should not have manifested in 10.1 codeline (the main codeline does have ColumnDscriptor constructor with >10 parameters).

        So, I am attaching the svn diff and svn stat for 10.1 codeline since it cleans up parameter passing to ColumnDescriptor but further investigation is required to see why the test program in JIRA fails on 10.1 codeline if it is supposed to fail only if the number of parameters are >10.

        Show
        Mamta A. Satoor added a comment - I am attaching the patch for 10.1 codeline after doing a merge from main and hand fixing some of the conflicts. (The derbyall test suite ran fine with jdk13) svn merge -r 413128:413129 https://svn.apache.org/repos/asf/db/derby/code/trunk/ One thing to note though is that in 10.1 codeline, the number of parameters to ColumnDescriptor constructors are 10 and not >10. So, from the bug diagnosis as mentioned in this JIRA entry, the bug should not have manifested in 10.1 codeline (the main codeline does have ColumnDscriptor constructor with >10 parameters). So, I am attaching the svn diff and svn stat for 10.1 codeline since it cleans up parameter passing to ColumnDescriptor but further investigation is required to see why the test program in JIRA fails on 10.1 codeline if it is supposed to fail only if the number of parameters are >10.
        Hide
        Kathey Marsden added a comment -

        Thanks Mamta!

        I commtted this patch:
        Date: Sat Jun 10 12:10:35 2006
        New Revision: 413354
        URL: http://svn.apache.org/viewvc?rev=413354&view=rev

        I manually merged the test which did not seem to merge properly. Please check it.

        I re-verified that the issue reproduced on the reported version without the patch.
        Then I verified that it did not reproduce with this patch, but it is indeed not clear why the issue manifested itself in 10.1 based on the JVM bug description which says you must have > 10 parameters.

        Even though the symptom is eleviated I think we should leave this issue unresolved until we have a better understanding of the JIT bug and why it manifested itself in 10.1 in the first place and determine if this is a reliable resolution to the issue.

        I will update this issue once the mystery is solved.

        Show
        Kathey Marsden added a comment - Thanks Mamta! I commtted this patch: Date: Sat Jun 10 12:10:35 2006 New Revision: 413354 URL: http://svn.apache.org/viewvc?rev=413354&view=rev I manually merged the test which did not seem to merge properly. Please check it. I re-verified that the issue reproduced on the reported version without the patch. Then I verified that it did not reproduce with this patch, but it is indeed not clear why the issue manifested itself in 10.1 based on the JVM bug description which says you must have > 10 parameters. Even though the symptom is eleviated I think we should leave this issue unresolved until we have a better understanding of the JIT bug and why it manifested itself in 10.1 in the first place and determine if this is a reliable resolution to the issue. I will update this issue once the mystery is solved.
        Hide
        Kathey Marsden added a comment -

        Taking off Patch Available as this has been committed to trunk and 10.1

        Show
        Kathey Marsden added a comment - Taking off Patch Available as this has been committed to trunk and 10.1
        Hide
        Kathey Marsden added a comment -

        I confirmed with the released IBM JDK Service release 2 that this issue does reproduce without Mamta's change and was fixed afterward. I am awaiting clarification on what is the minimum number of parameters to trigger the JVM bug but since this is a compile time issue I think that Mamta's fix is enough to resolve DERBY-1327 in both trunk and 10.1 as I do not think there are any runtime configurations that could cause different behaviour. Below is a summary of this issue from a user perspective.

        PROBLEM

        After creating a table, with JIT engaged, identity column start with values for any column in the database gets changed to an incorrect value.

        SYMPTOM

        The symptom is typically that an insert to the table will fail with the Exception:
        SQLSTATE 22001: "A truncation error was encountered trying to shrink ... to length 12."

        CAUSE
        A JIT issue in IBM JDK 1.5 Service Release 2 can cause corruption of the start with value. The affected version is:
        java version "1.5.0"
        Java(TM) 2 Runtime Environment, Standard Edition (build pwi32dev-20060511 (SR2))
        IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 Windows XP x86-32 j9vmwi3223-20060504 (JIT enabled)
        J9VM - 20060501_06428_lHdSMR
        JIT - 20060428_1800_r8
        GC - 20060501_AA)
        JCL - 20060511a

        SOLUTION:

        A fix to resolve the Derby symptom is available in 10.1 builds after 10.1.2.5.413354 and will be included in the Derby 10.1.3.0 release.

        Once corruption has occurred there is no way in 10.1 to correct the START WITH value. The only way to recover date is to recreate the database and use export/import to transfer the data.

        WORKAROUND:
        If users wish to wait for the 10.1.3.0 official release, The following JIT options can be specified for the JVM to prevent corruption in identity columns but this does allow recovery after the problem has occured.

        -Xjit:exclude=

        {org/apache/derby/impl/sql/execute/CreateTableCon stantAction.executeConstantAction\(Lorg/apache/derby/iapi/sql/Ac tivation\;\)V}
        Show
        Kathey Marsden added a comment - I confirmed with the released IBM JDK Service release 2 that this issue does reproduce without Mamta's change and was fixed afterward. I am awaiting clarification on what is the minimum number of parameters to trigger the JVM bug but since this is a compile time issue I think that Mamta's fix is enough to resolve DERBY-1327 in both trunk and 10.1 as I do not think there are any runtime configurations that could cause different behaviour. Below is a summary of this issue from a user perspective. PROBLEM After creating a table, with JIT engaged, identity column start with values for any column in the database gets changed to an incorrect value. SYMPTOM The symptom is typically that an insert to the table will fail with the Exception: SQLSTATE 22001: "A truncation error was encountered trying to shrink ... to length 12." CAUSE A JIT issue in IBM JDK 1.5 Service Release 2 can cause corruption of the start with value. The affected version is: java version "1.5.0" Java(TM) 2 Runtime Environment, Standard Edition (build pwi32dev-20060511 (SR2)) IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 Windows XP x86-32 j9vmwi3223-20060504 (JIT enabled) J9VM - 20060501_06428_lHdSMR JIT - 20060428_1800_r8 GC - 20060501_AA) JCL - 20060511a SOLUTION: A fix to resolve the Derby symptom is available in 10.1 builds after 10.1.2.5.413354 and will be included in the Derby 10.1.3.0 release. Once corruption has occurred there is no way in 10.1 to correct the START WITH value. The only way to recover date is to recreate the database and use export/import to transfer the data. WORKAROUND: If users wish to wait for the 10.1.3.0 official release, The following JIT options can be specified for the JVM to prevent corruption in identity columns but this does allow recovery after the problem has occured. -Xjit:exclude= {org/apache/derby/impl/sql/execute/CreateTableCon stantAction.executeConstantAction\(Lorg/apache/derby/iapi/sql/Ac tivation\;\)V}
        Hide
        Kathey Marsden added a comment -

        Verified fix

        Show
        Kathey Marsden added a comment - Verified fix
        Hide
        Kathey Marsden added a comment -

        The core JVM issue can be tracked with IBM APAR APAR number IY86935.

        Show
        Kathey Marsden added a comment - The core JVM issue can be tracked with IBM APAR APAR number IY86935.
        Hide
        Kathey Marsden added a comment -
        Show
        Kathey Marsden added a comment - APAR link for tracking: http://www-1.ibm.com/support/docview.wss?uid=swg1IY86935

          People

          • Assignee:
            Mamta A. Satoor
            Reporter:
            Kathey Marsden
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development