I don't think we have the resources to build and maintain automated tests for other DBMS implementations;
it's hard enough to maintain sophisticated SQL tests for Derby.
Mostly, my idea is that the test suites that exist for other databases (H2, MySQL, Postgres, etc.) may be a rich source of ideas for how to extend and improve the Derby test suites, particularly in the area of finding unusual SQL that different DBMS implementations handle differently (SQL is a very rich language with many variations on statements).
So the goal is to improve the Derby test suites, not to build cross-database test suites.
Note that there are other projects (e.g., Apache OpenJPA) which do in fact build sophisticated cross-database test suites, so such things do exist. They just aren't part of the Derby community's test suites.
If we add a test to the Derby suite, there are two situations:
1) The test uses non-standard SQL. In that case, the test should simply verify that Derby gives a reasonable error message. It would be useful and helpful if the test included comments describing how the non-standard SQL behaves on some other DBMS, but that's not required. It would also be useful and helpful to update the Derby wiki in such a case to describe the implementation differences.
2) The test uses standard SQL; we found the test in another DBMS's open test suite, determined that Derby doesn't support that SQL properly, opened a bug against Derby to describe the problem we found, and added a test
case to the Derby test suite. Here's an interesting example of such a case: https://issues.apache.org/jira/browse/DERBY-4558 The goal is, can we find more?
This is all somewhat hypothetical, as first we need to find some test suites for other database systems that are worth running against Derby