Attaching d5947-5a-row-count-stats.diff, which moves the row count
statistics from generated code to GenericPreparedStatement.
It was trickier than I had expected to get the tests to run cleanly
with the patch. I saw intermittent failures in the upgrade tests. They
failed because they triggered an assert in store when booting the
database for hard upgrade:
Caused by: org.apache.derby.shared.common.sanity.AssertFailure: ASSERT FAILED initSlotTable consistency check failed: slot 0 minimumRecordSize = 12 totalSpace = 12 recordPortionLength = 8 reservedCount = 4
It only happened occasionally, and it looked like the order in which
the test cases ran determined whether or not the test run succeeded. I
managed to isolate it further and reliably reproduce it using the
1) Create an empty database with Derby 10.4.2.0 or earlier
2) Boot the database with trunk+patch (soft-upgrade) and create two
tables and a trigger:
create table a(x int);
create table b(y int);
create trigger t after update on b
referencing new as n for each row
insert into a values (n.y);
3) Boot the database once with any Derby version 10.6.2.1 or earlier
4) Boot the database again with trunk, and see it fail with the above
mentioned assert failure
The failing assert was added in
DERBY-4577. And indeed it only
reproduces if the downgrade step (3) is performed with a version that
So what seems to be happening, is the following:
When the trigger is created in step (2) with trunk+patch, the trigger
plan is stored in a column in SYS.SYSSTATEMENTS. The plan is large
enough to make the row overflow to another page. When downgrading the
database in step (3), the old Derby version will go through all rows
in SYS.SYSSTATEMENTS and set the column that holds compiled plans to
NULL. Apparently, when shrinking the row, the old version runs into
the condition that caused
DERBY-4577, and ends up setting aside too
little space for future expansion of the row. When subsequently
booting the database again with trunk, the assert in StoredPage
recognizes that this has happened, and fails.
If a version that includes the fix for
DERBY-4577 is used in step 3,
the corruption doesn't happen, and trunk doesn't complain when booting
the database later.
Although I've only seen the assert failure with the patch, I don't
think it's a problem introduced by the patch. It's just that the new
and smaller trigger plans happen to make the SYSSTATEMENT rows
produced by these test cases of the exact right size to hit
DERBY-4577. Even without the patch, it might be possible to construct
a SYSSTATEMENTS table that trips over the same problem.
The patch works around this problem by forcing the problematic test
cases to run in a known good order. I've run the full upgrade test
suite successfully ten times with the workaround. Without the
workaround it would fail approximately every other run (with Java 7,
that is, where the test ordering varies between runs).
Finally, a description of what the patch does:
Removed code added by the 1a patch for manipulating static fields and
static initializers for the row count statistics.
Added fields to hold execution count, result set row count and stale
plan interval for a statement.
The fields were put in an inner class instead of directly inside
GenericPreparedStatement. This was done because triggers clone the
GenericPreparedStatement on each execution, so the statistics needed
to be shared among multiple GPS instances. For all other statements
than triggers, the statistics instance is private to one GPS instance.
Also, since the lifespan of a GPS instance is longer than that of the
generated activation class, code to reset the statistics on
recompilation was added.
Interface methods that allowed calls to the new methods in
GenericPreparedStatement from BaseActivation.
Removed inner class that held row count statistics.
Made shouldWeCheckRowCounts() and informOfRowCount() save the
statistics in the GenericPreparedStatement instead of the now removed
Removed a method that's no longer used.
Made the tests run in a fixed order, for reasons explained above.