Issue Details (XML | Word | Printable)

Key: DERBY-85
Type: Bug Bug
Status: Closed Closed
Resolution: Fixed
Priority: Major Major
Assignee: Dyre Tjeldvoll
Reporter: A B
Votes: 0
Watchers: 0
Operations

If you were logged in you would be able to see more operations.
Derby

NPE when creating a trigger on a table and default schema doesn't exist.

Created: 01/Dec/04 02:05 AM   Updated: 24/May/06 04:38 AM
Return to search
Component/s: SQL
Affects Version/s: 10.0.2.0
Fix Version/s: 10.1.3.1, 10.2.1.6

Time Tracking:
Not Specified

File Attachments:
  Size
File Licensed for inclusion in ASF works derby-85.diff 2006-01-25 12:02 AM Dyre Tjeldvoll 4 kB
File Licensed for inclusion in ASF works derby-85.stat 2006-01-25 12:02 AM Dyre Tjeldvoll 0.5 kB
File Licensed for inclusion in ASF works derby-85_2.diff 2006-02-13 04:38 PM Dyre Tjeldvoll 3 kB
File Licensed for inclusion in ASF works derby-85_2.stat 2006-02-13 04:38 PM Dyre Tjeldvoll 0.3 kB
Text File Licensed for inclusion in ASF works derbyall_report.merge-to-10.1.txt 2006-04-28 07:18 PM Dyre Tjeldvoll 19 kB
Text File Licensed for inclusion in ASF works derbyall_report.txt 2006-01-25 12:02 AM Dyre Tjeldvoll 26 kB
Text File Licensed for inclusion in ASF works derbyall_report_2.txt 2006-02-13 04:38 PM Dyre Tjeldvoll 1.94 MB

Resolution Date: 29/Apr/06 06:27 AM


 Description  « Hide
BACKGROUND:

When connecting to a Derby db with a user id and password, the default schema is USER. For example, if I connect with:

ij> connect 'jdbc:derby:myDB;user=someUser;password=somePwd';

then the default schema is "SOMEUSER".

PROBLEM:

It turns out that if a table t1 exists in a non-default schema and the default schema (in this case, "SOMEUSER") doesn't exist yet (because no objects have been created in that schema), then attempts to create a trigger on t1 using its qualified name will lead to a null pointer exception in the Derby engine.

REPRO:

In ij:

-- Create database with default schema "SOMEUSER".
ij> connect 'jdbc:derby:myDB;create=true;user=someUser;password=somePwd';

-- Create table t1 in a non-default schema; in this case, call it "ITKO".
ij> create table itko.t1 (i int);
0 rows inserted/updated/deleted

-- Now schema ITKO exists, and T1 exists in schema ITKO, but default schema SOMEUSER does NOT exist, because we haven't created any objects in that schema yet.

-- So now we try to create a trigger in the ITKO (i.e. the non-default) schema...
ij> create trigger trig1 after update on itko.t1 for each row mode db2sql select * from sys.systables;
ERROR XJ001: Java exception: ': java.lang.NullPointerException'.

A look at the derby.log file shows the stack trace given below. In a word, it looks like the "compilation schema" field of SYS.SYSTRIGGERS isn't getting set, and so it ends up being null. That causes the NPE in subsequent processing...

java.lang.NullPointerException
at org.apache.derby.impl.sql.catalog.SYSSTATEMENTSRowFactory.makeSYSSTATEMENTSrow(SYSSTATEMENTSRowFactory.java:200)
at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.addSPSDescriptor(DataDictionaryImpl.java:2890)
at org.apache.derby.impl.sql.execute.CreateTriggerConstantAction.createSPS(CreateTriggerConstantAction.java:354)
at org.apache.derby.impl.sql.execute.CreateTriggerConstantAction.executeConstantAction(CreateTriggerConstantAction.java:258)
at org.apache.derby.impl.sql.execute.MiscResultSet.open(MiscResultSet.java:56)
at org.apache.derby.impl.sql.GenericPreparedStatement.execute(GenericPreparedStatement.java:366)
at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(EmbedStatement.java:1100)
at org.apache.derby.impl.jdbc.EmbedStatement.execute(EmbedStatement.java:509)
at org.apache.derby.impl.jdbc.EmbedStatement.execute(EmbedStatement.java:467)
at org.apache.derby.impl.tools.ij.ij.executeImmediate(ij.java:299)
at org.apache.derby.impl.tools.ij.utilMain.doCatch(utilMain.java:433)
at org.apache.derby.impl.tools.ij.utilMain.go(utilMain.java:310)
at org.apache.derby.impl.tools.ij.Main.go(Main.java:210)
at org.apache.derby.impl.tools.ij.Main.mainCore(Main.java:176)
at org.apache.derby.impl.tools.ij.Main14.main(Main14.java:56)
at org.apache.derby.tools.ij.main(ij.java:60)

 All   Comments   Work Log   Change History   Subversion Commits      Sort Order: Ascending order - Click to sort in descending order
Dyre Tjeldvoll added a comment - 07/Jun/05 11:00 PM
Looks like the oid for the default schema is null until a row is
inserted in a table in the default schema. This oid is referenced when
creating a trigger, even if that trigger doesn't touch the default schema.

One can work around it by having a non-empty table in the default
schema:

s.execute("create table itko.t1 (i int)");

// Workaround
s.execute("create table def (i int)");
s.execute("insert into def values (0), (1), (2)");

s.execute("create trigger ikto.trig1 after update on itko.t1 for each row mode db2sql select * from sys.systables");

I THINK that when creating a trigger, the code needs a "compile schema" and it
assumes that it is safe to use the default schema, which it isn't in
this case.

So I guess you could attack this problem in (at least) two ways:
1) Make the trigger code use a schema that is known to be ok
2) Ensure that the default schema always is valid.

Dyre Tjeldvoll added a comment - 14/Jun/05 06:38 AM
Simpler workaround provided by Daniel John Debrunner <djd@debrunners.com> in <42AB428F.1000002@debrunners.com>:
Or simpler, just create the schema using CREATE SCHEMA.
 
CREATE SCHEMA user
SET SCHEMA user

CREATE TRIGGER...

Dyre Tjeldvoll added a comment - 14/Jun/05 06:41 AM
Full example provided by Mamta Satoor <msatoor@gmail.com> in <d9619e4a05061122157335250d@mail.gmail.com>:

Actually, in addition to Dan's suggestion of CREATE SCHMEA, you have to next
do SET SCHEMA, and then you don't need to create a dummy table in the
schema. To be specific, following eg worked in ij
ij> connect
'jdbc:derby:c:/dellater/db1;create=true;user=mamta;password=mamta';
ij> create table itko.t1 (i int);
0 rows inserted/updated/deleted
ij> create schema mamta;
0 rows inserted/updated/deleted
ij> set schema mamta;
0 rows inserted/updated/deleted
ij> create trigger itko.trig1 after update on itko.t1 for each row mode
db2sql select * from sys.systables;
0 rows inserted/updated/deleted
ij> exit;

Dyre Tjeldvoll added a comment - 15/Jun/05 12:46 AM
The javadoc for the CreateTriggerConstantAction constructor seems to confirm that the default schema is used if no other compile schema is provided:

spsCompSchemaId - the compilation schema for the action and when spses. If null, will be set to the current default schema

The code confirms this:

return getGenericConstantActionFactory().getCreateTriggerConstantAction(
triggerSchemaDescriptor.getSchemaName(),
getRelativeName(),
triggerEventMask,
isBefore,
isRow,
isEnabled,
triggerTableDescriptor,
(UUID)null, // when SPSID
whenText,
(UUID)null, // action SPSid
actionText,
(actionCompSchemaId == null) ?
compSchemaDescriptor.getUUID() :
actionCompSchemaId,
(Timestamp)null, // creation time
referencedColInts,

I tried to add:
if (SanityManager.DEBUG) {
SanityManager.ASSERT((actionCompSchemaId != null ||
compSchemaDescriptor.getUUID()!=
null),
"No valid compile schema");
}

return getGenericConstantActionFactory().getCreateTriggerConstantAction(...

and this shows that both are null in this case:

dt136804@atum10~/java$ run-app.sh Try
Try starting in embedded mode.
TRACE: Class.forName(driver).newInstance(); @Try.java.m4:89
TRACE: conn = DriverManager.getConnection(protocol + db + ";create=true", props); @Try.java.m4:105
TRACE: conn.setAutoCommit(false); @Try.java.m4:107
TRACE: Statement s = conn.createStatement(); @Try.java.m4:113
s=org.apache.derby.impl.jdbc.EmbedStatement@126e85f @Try.java.m4:114
conn=0 @Try.java.m4:115
TRACE: s.execute("create table itko.t1 (i int)"); @Try.java.m4:121
TRACE: s.execute("create schema USER1"); @Try.java.m4:122
TRACE: s.execute("create trigger ikto.trig1 after update on itko.t1 for each row mode db2sql select * from sys.systables"); @Try.java.m4:128
exception thrown:
SQL Exception: Java exception: 'ASSERT FAILED No valid compile schema: org.apache.derby.iapi.services.sanity.AssertFailure'.
SQLState: XJ001 MessageId: XJ001.U
org.apache.derby.iapi.services.sanity.AssertFailure: ASSERT FAILED No valid compile schema
        at org.apache.derby.iapi.services.sanity.SanityManager.ASSERT(SanityManager.java:121)
        at org.apache.derby.impl.sql.compile.CreateTriggerNode.makeConstantAction(CreateTriggerNode.java:728)
        at org.apache.derby.impl.sql.GenericStatement.prepMinion(GenericStatement.java:459)
        at org.apache.derby.impl.sql.GenericStatement.prepare(GenericStatement.java:107)
        at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.prepareInternalStatement(GenericLanguageConnectionContext.java:688)
        at org.apache.derby.impl.jdbc.EmbedStatement.execute(EmbedStatement.java:501)
        at org.apache.derby.impl.jdbc.EmbedStatement.execute(EmbedStatement.java:475)
        at Try.go(Try.java:128)
        at Try.main(Try.java:73)
null
Try finished

Dyre Tjeldvoll added a comment - 20/Jun/05 04:57 AM
The patch appears to fix the problem and passes derbyall (see attached derbyall_report.txt), but I'm a bit unsure if this is the best way to fix the problem.
It would be really nice if a committer could look at it, and perhaps give some feedback.

Some comments to the patch:
- The compile schema is needed in the CREATE TRIGGER statement in order to fill in column 8 of the SYSSTATEMENTS table
- The CREATE TRIGGER code uses LanguageConnectionContext.getDefaultSchema() as a fallback when no other compile schema is specified. The object returned by this method has a null oid unless one has done SET SCHEMA. But a SchemaDescriptor containing a valid oid can be obtained by calling
getSchemaDescriptorForCreate(DataDictionary, Activation, String). The patch uses this method to get the oid as the ultimate fallback.

Dyre Tjeldvoll added a comment - 21/Jun/05 01:35 AM
New diff without tabs.

Dyre Tjeldvoll added a comment - 21/Jun/05 04:18 PM
Looking at the code again, I wondering if it would be better to use DataDictionary.getSchemaDescriptor(...) rather than DDLConstantAction.getSchemaDescriptorForCreate(...) since we just want to look at the Descriptor object...

Rick Hillegas added a comment - 01/Nov/05 04:50 AM
The patch itself looks good. I'm running derbyall now. The patch, however, needs to add a regression test case to one of the language tests to verify that the bug is fixed.

Rick Hillegas added a comment - 01/Nov/05 06:22 AM
Derbyall passed. When a new patch is submitted including a regression test, I'll just run the appropriate suite.

Kathey Marsden added a comment - 11/Nov/05 07:41 AM
What is the status of this fix? From the comments it looked like it was pretty close to complete.

Thanks

Kathey

Dyre Tjeldvoll added a comment - 17/Nov/05 12:36 AM
Status? If you look at the dates you'll see that I have not looked at this for about 5 months, so I don't really know. I'm actually surprised that Rick was able to apply the patch without problems.

If am supposed to create a regression test for this; is there any chance that I can do that as JUnit test? And, can someone please comment on my question from 21 June? About which method to use to obtain a schema descriptor?

Rick Hillegas added a comment - 17/Nov/05 01:28 AM
I don't think I understand the June 21 question about which schema descriptor method to call. DDLConstantAction.getSchemaDescriptorForCreate() will persistently create the schema if it doesn't already exist. I think you need to do that before you create any objects in the schema. I don't think that the DataDictionary methods, by themselves, will create any persistent objects. So the method you are calling looks like the right one to me.

Dyre Tjeldvoll added a comment - 23/Nov/05 12:43 AM
This is my attempt at creating a JUnit test for DERBY-85. It is currently just an ordinary JUnit test, I have not done anything to try to integrate it with the existing
test harness (I could not find any info on how to do that). I have run the test using junit.swingui.TestRunner and that seems to work. If someone would like to review it that would be great.

Dyre Tjeldvoll added a comment - 01/Dec/05 01:34 AM
New patch with Junit test case.

Note that I had to modify the SecurityManager policy to grant more privileges to derbyTesting.jar and junit.jar
Anyone interested SecurityManager issues may want to review those changes carefully.

Also note that the current patch does NOT use DDLConstantAction.getSchemaDescriptorForCreate() , but accesses the dictionary directly. This works fine because the trigger is NOT created in this schema. You ONLY need the oid of the default schema to fill in a column in the SYSSTATEMENTS table, (described in more detail in earlier comments).

lang/closed.java failed when I ran derbyall, but I don't think this is related to the patch (see DERBY-734).

Dyre Tjeldvoll added a comment - 25/Jan/06 12:02 AM
New patch based on the latest trunk. The patch is as before, but the test case is now a simple ij-test, that does not cause any SecurityManager problems. Hopefully this issue can be committed soon, (if I scratch this itch any more I'm going to get gangrene).

Satheesh Bandaram added a comment - 10/Feb/06 05:01 PM
Looks like this patch keeps dropping off the radar... Dyre, why exactly are you adding a new test script with one simple test? Could it be possible to add this test to existing trigger tests? It is possible to get a connection to 'someUser' in the middle of an existing test.

Dyre Tjeldvoll added a comment - 10/Feb/06 06:39 PM
Hi Sateesh, thank you so much for taking the time to look at the patch! Is it really possible to get a different connection inside an ij-script? Wow, I did not know that, how do you do that? Do you have an example?

Which test script should this go into? I find the following trigger-related test scripts in the lang directory:

triggerBeforeTrig.sql triggerRefClause.sql
triggerGeneral.sql triggerRefClause_app.properties
triggerGeneral_app.properties triggerStream.java
triggerGeneral_derby.properties triggerStream_app.properties
triggerRecursion.sql triggerStream_derby.properties
triggerRecursion_derby.properties

My money would be on trggerGeneral.sql, but since the "common sense" approach keeps letting me down, I thought I had better ask :)

Dyre Tjeldvoll added a comment - 13/Feb/06 04:38 PM
New patch addressing Sateesh concerns about the regression test being a separate test script. Please review. Thanks.

Satheesh Bandaram added a comment - 16/Feb/06 07:49 AM
Thanks for addressing review comments. I have submitted this fix. Please resolve the fix (with fix version of 10.2) and then close after verifying the fix.

Sending java\engine\org\apache\derby\impl\sql\execute\CreateTriggerConstantAction.java
Sending java\testing\org\apache\derbyTesting\functionTests\master\triggerGeneral.out
Sending java\testing\org\apache\derbyTesting\functionTests\tests\lang\triggerGeneral.sql
Transmitting file data ...
Committed revision 378109.

Dyre Tjeldvoll added a comment - 16/Feb/06 06:20 PM
Committed by Sateesh Bandaram

Sending java\engine\org\apache\derby\impl\sql\execute\CreateTriggerConstantAction.java
Sending java\testing\org\apache\derbyTesting\functionTests\master\triggerGeneral.out
Sending java\testing\org\apache\derbyTesting\functionTests\tests\lang\triggerGeneral.sql
Transmitting file data ...
Committed revision 378109.

Dyre Tjeldvoll added a comment - 16/Feb/06 06:22 PM
Verified

Satheesh Bandaram added a comment - 28/Apr/06 09:35 AM
I think it would be great to port this fix to 10.1 branch. I have seen some users hit this problem. Dyre, do you happen to have itch to port this to 10.1?

Dyre Tjeldvoll added a comment - 28/Apr/06 07:08 PM
Setting fixin for merge to 10.1.3

Dyre Tjeldvoll added a comment - 28/Apr/06 07:18 PM
svn merge -r 378108:378109 trunk 10.1@397791M was clean

Ran derbyall after the merge with 3 failures, which I don't think are related (see attached report file for details).

Andrew McIntyre added a comment - 29/Apr/06 06:27 AM
Committed to 10.1 branch with revision 398036.

A B added a comment - 24/May/06 04:38 AM
Verified fix and it has been ported to 10.1, so I'm closing the issue.