[DERBY-5671] NsTest does not run on trunk do multiple issues stemming from concurrency improvements

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 10.9.1.0
• Component/s: SQL
• Labels: derby_backport_reject_10_8
• Issue & fix info: High Value Fix
• Bug behavior facts: Regression
• Assignee: Unassigned
• Reporter: Katherine Marsden
• Created: 25/Mar/12 18:53
• Updated: 12/Jul/13 16:50
• Resolved: 19/Apr/12 02:12

Description

As I understand it at least since September 30 of last year, the system test NsTest has been broken on trunk. In these six months the test has not been runnable, so we do not know if new issues have been introduced with sequence generators or most importantly with auto-increment columns that are now based on them, which many, many applications rely upon. Even if the known problems are fixed later in the 10.9 release cycle and new problems are exposed, we won't be able to go back to any point in time to discover when they might be released.

In 10.8 we coped with this problem by backing out the concurrency improvements (DERBY-5448) pending fixes for DERBY-5422, DERBY-5454, DERBY-5430. Currently none of those issues have been assigned. Since this has been going on now for six months, I think we urgently need to stabiliize auto-increment columns and get this test running again on trunk. I can see three possible options.
1) Someone with interest assign themselves to these issues and make significant progress over the next few weeks.
2) Make the concurrency improvements optional with a property which defaults to false (I don't know if this is practical)
3) Back the concurrency performance improvements out of trunk until these issues have been resolved and the change can be resubmitted.

I realize that NsTest is not the easiest test to work with but it does seem to have found serious problems with generated columns that I think users are likely to hit. In the past, a similiar disregard for mailjdbc exposing a corruption issue meant that we actually released a bad corruption issue that I know hit many users of Derby before we addressed it. Autoincrement is widely, widely, used. We need to get it stabilized and the test running on trunk. Although the system tests are not particularly easy to deal with, they are all we have and they do find issues.

Issue Links

• is related to: DERBY-5422 IndexOutOfBoundsException followed by NullPointerException in cleanup during system.nstest
• is related to: DERBY-5430 Embedded nstest shows ERROR 40001: A lock could not be obtained due to a deadlock, in nstest
• is related to: DERBY-5448 In the 10.8 branch, back out the concurrency improvements introduced by using SequenceUpdaters to allocate identity values.
• is related to: DERBY-5454 ERROR 40001 deadlock in nstest on select max(serialkey)
• is related to: DERBY-5687 Back out the concurrency improvements for identity columns introduced by derby-4437
• relates to: DERBY-5495 Master issue to track fixes to sequence generators