Another way to get the test case to pass, also on Java versions earlier than 7, is to call the URLConnection.setDefaultUseCaches() method first to disable caching of the jar files opened by URLClassLoader. That seems to make gc capable of freeing the file handle after the context class loader has been set to null at the end of the test case. An explicit call to System.gc() is needed before trying to delete the jar file, though, in order to make sure gc has run and released the file handle.
This latter approach is probably too brittle to use in the regression tests, since System.gc() doesn't guarantee that all garbage is cleaned up. But I think these results support Dan's initial analysis that the file handle is kept open by the Java platform itself and not by Derby. I also think it means the solution for this problem must be implemented in the application layer (applications could either close the URLClassLoader when they are done with it, or disable caching of jar files). Having Derby close the URLClassLoader on shutdown is not an option, I think, as it is not possible for Derby to determine whether or not the application will still need the class loader after the Derby engine has been shut down.