As OODT continues to mature, i think we are getting to the point where we need some kind of release integration check infrastructure that nominally will bring up all the components and execute some test runs on them... at first these tests will be run manually by engineers before each OODT release... thus encouraging them to automate them ... currently there is no way to prove a released version of OODT is any more a 'blessed' version of the code than any given checked out revision. I think having 'blessed' stable releases is important.
I'm thinking of creating 2 google docs. The first being a word document enumerating the scenarios which should be run. The second being a spreadsheet with a tab for each release enumerating which scenarios where run and whether or not passed (with a column for bug ID created if scenario fails). As each scenario gets automated, it can then be mark as such and only the none automated scenario will continue to be executed by engineers.