Derby
  1. Derby
  2. DERBY-230

"Schema already exists" when creating a table

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 10.0.2.0, 10.0.2.1, 10.0.2.2, 10.1.1.0
    • Fix Version/s: 10.1.1.0
    • Component/s: SQL
    • Labels:
      None

      Description

      When running a multithreaded program where several threads in parallell create tables in a schema that is not explicitly created, one often get the following exception:

      ERROR X0Y68: Schema 'TESTSCHEMA' already exists.
      at org.apache.derby.iapi.error.StandardException.newException(StandardException.java:322)
      at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.duplicateDescriptorException(DataDictionaryImpl.java:1512)
      at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.addDescriptorNow(DataDictionaryImpl.java:1496)
      at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.addDescriptor(DataDictionaryImpl.java:1478)
      at org.apache.derby.impl.sql.execute.CreateSchemaConstantAction.executeConstantAction(CreateSchemaConstantAction.java:147)
      at org.apache.derby.impl.sql.execute.DDLConstantAction.getSchemaDescriptorForCreate(Unknown Source)
      at org.apache.derby.impl.sql.execute.CreateTableConstantAction.executeConstantAction(CreateTableConstantAction.java:213)
      at org.apache.derby.impl.sql.execute.MiscResultSet.open(MiscResultSet.java:56)
      at org.apache.derby.impl.sql.GenericPreparedStatement.execute(GenericPreparedStatement.java:366)
      at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(EmbedStatement.java:1108)
      at org.apache.derby.impl.jdbc.EmbedStatement.execute(EmbedStatement.java:517)
      at org.apache.derby.impl.jdbc.EmbedStatement.execute(EmbedStatement.java:475)
      at derbytest.Main$CreateTable.run(Main.java:42)
      at java.lang.Thread.run(Thread.java:595)

      A program that reproduces this bug will be attached.

      1. derby-230c.diff
        10 kB
        Øystein Grøvlen
      2. Main.java
        5 kB
        Øystein Grøvlen

        Activity

        Hide
        Øystein Grøvlen added a comment -

        Attaching a new version of the patch since the test with 200 threads once in a while experienced locking timeouts.
        Hence, I am going back to 100 threads. Also removed some unintentional property settings.

        Show
        Øystein Grøvlen added a comment - Attaching a new version of the patch since the test with 200 threads once in a while experienced locking timeouts. Hence, I am going back to 100 threads. Also removed some unintentional property settings.
        Hide
        Øystein Grøvlen added a comment -

        Attaching a new version of the patch where I have increased the probability of the test reproducing the bug by increasing the number of threads to 200. It now fails 16 out of 20 times on a single-CPU machine. Increasing the number of threads further seems to increase the execution time very much. Considering that when running derbyall, the test is run three times in different frameworks, the probability of the test suite detecting the rebirth of this bug should be more than 98%. On a 2-CPU machine the test failed 20 out of 20 times.

        Show
        Øystein Grøvlen added a comment - Attaching a new version of the patch where I have increased the probability of the test reproducing the bug by increasing the number of threads to 200. It now fails 16 out of 20 times on a single-CPU machine. Increasing the number of threads further seems to increase the execution time very much. Considering that when running derbyall, the test is run three times in different frameworks, the probability of the test suite detecting the rebirth of this bug should be more than 98%. On a 2-CPU machine the test failed 20 out of 20 times.
        Hide
        Øystein Grøvlen added a comment -

        Here is a new version of the patch which addresses the review comments by Army and Dan:

        • The new test, ConcurrentImplicitCreateSchema.java, has also been added to the client/server testsuites.
        • startJBMS() is used to get all connections

        derbyall has been run with one known error, DERBY-320

        Show
        Øystein Grøvlen added a comment - Here is a new version of the patch which addresses the review comments by Army and Dan: The new test, ConcurrentImplicitCreateSchema.java, has also been added to the client/server testsuites. startJBMS() is used to get all connections derbyall has been run with one known error, DERBY-320
        Hide
        Øystein Grøvlen added a comment -

        I have attached a patch to fix this bug. Please, review it.

        The patch contains changes to the following files:
        M java/engine/org/apache/derby/impl/sql/execute/DDLConstantAction.java
        The fix as previously described in this JIRA report.

        A java/testing/org/apache/derbyTesting/functionTests/tests/lang/ConcurrentImplicitCreateSchema.java
        A new test that, before the fix was applied, reproduced the problem almost every time on a multi-processor computer and once in a while on a single-threaded computer.

        A java/testing/org/apache/derbyTesting/functionTests/master/ConcurrentImplicitCreateSchema.out
        The master file for the test.

        M java/testing/org/apache/derbyTesting/functionTests/suites/derbylang.runall
        The test has been included in the derbylang suite.

        I have run the derbyall testsuite, and all tests passed except derbynet/testconnection.java which also failed without this patch (see DERBY-303).

        Show
        Øystein Grøvlen added a comment - I have attached a patch to fix this bug. Please, review it. The patch contains changes to the following files: M java/engine/org/apache/derby/impl/sql/execute/DDLConstantAction.java The fix as previously described in this JIRA report. A java/testing/org/apache/derbyTesting/functionTests/tests/lang/ConcurrentImplicitCreateSchema.java A new test that, before the fix was applied, reproduced the problem almost every time on a multi-processor computer and once in a while on a single-threaded computer. A java/testing/org/apache/derbyTesting/functionTests/master/ConcurrentImplicitCreateSchema.out The master file for the test. M java/testing/org/apache/derbyTesting/functionTests/suites/derbylang.runall The test has been included in the derbylang suite. I have run the derbyall testsuite, and all tests passed except derbynet/testconnection.java which also failed without this patch (see DERBY-303 ).
        Hide
        Øystein Grøvlen added a comment -

        According to Dan Debrunner alternative 1 is not correct since there are other ways schemas come into existence than through this method. Hence, I will fix this as suggested in the second alternative by catching exceptions with messageId == SQLState.LANG_OBJECT_ALREADY_EXISTS.

        Show
        Øystein Grøvlen added a comment - According to Dan Debrunner alternative 1 is not correct since there are other ways schemas come into existence than through this method. Hence, I will fix this as suggested in the second alternative by catching exceptions with messageId == SQLState.LANG_OBJECT_ALREADY_EXISTS.
        Hide
        Øystein Grøvlen added a comment -

        As far as I can understand, the problem is in DDLConstantAction.getSchemaDescriptorForCreate() which first tries to
        get a SchemaDescriptor and if none exists will create one. What probably happens is that a thread detects that the schema does not exist, but when it tries to create the SchemaDescriptor, another thread has already done that. There is at least two possible ways to fix this:

        1. Make getSchemaDescriptorForCreate() synchronized. (I have tested this and it solves the problem).

        2. Ignore the "Schema already exists" exception from executeConstantAction. The subsequent getSchemaDescriptor will then get the SchemaDescriptor created by the other thread.

        Show
        Øystein Grøvlen added a comment - As far as I can understand, the problem is in DDLConstantAction.getSchemaDescriptorForCreate() which first tries to get a SchemaDescriptor and if none exists will create one. What probably happens is that a thread detects that the schema does not exist, but when it tries to create the SchemaDescriptor, another thread has already done that. There is at least two possible ways to fix this: 1. Make getSchemaDescriptorForCreate() synchronized. (I have tested this and it solves the problem). 2. Ignore the "Schema already exists" exception from executeConstantAction. The subsequent getSchemaDescriptor will then get the SchemaDescriptor created by the other thread.

          People

          • Assignee:
            Øystein Grøvlen
            Reporter:
            Øystein Grøvlen
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development