rather then having "@Weekly" or "@Slow" (or even "@Nightly") annotations, we should just use an "@Weight(Integer)" annotation indicating how "heavy" a test is
But I'm not sure I agree with this. Most tests have completely different runtime based on what configurations they get. Sometimes, rarely, we apply totally crazy parameters that make them go "nuts". Because of this we amortize the cost of checking expensive corner cases (like termIndexInterval=1, horrible params that cause a lot of merging, etc) across the different tests. I think this is good, it really finds corner case bugs. So if we had this weight annotation, i have no clue what we would assign it to anything.
Because you take a test that does a lot of skipping/conjunctions, and if it gets memorycodec it might be take a terribly long time (since memorycodec doesn't support skipping), but otherwise it doesnt really care about termsindexinterval. On the other hand you take a test that is torturing the terms dictionary, and it doesnt care if you give it memorycodec or a shitty skipinterval, but you give it a crazy termsindexinterval and it spends a lot of time next'ing thru terms, etc.
Because of this, all of the tests behave in totally different ways that you cant really assign a weight to, e.g. take a look at the history of test times
for this test: