This is an idea Doug mentioned on
This patch adds new targets to build.xml to automatically download the junit tests from a previous Lucene release and run them against the current core.
Execute tests like this:
ant -Dtag=lucene_2_4_0 test-tag
It will create a new directory tags/lucene_2_4_0 and fetch the tests from the svn repository and run them.