I see this tripping somebody up down the road.
I do too, hence "Should we do this?".
But for the tweak-compile-test cycle you describe, you can just use ant test, right? In other words, your concern isn't lack of support for that pattern, but rather that ill-informed people would be lulled into thinking they're testing up-to-date compilations of source code rather than previous compilations. One way to address this could be to reduce advertisement of the target, e.g. removing the target's description so that it doesn't show up when a user runs ant -p.
My opinion (and why I did the work): we should provide both options, and educate users about the difference.
the use cases for this
I think the main use case is for external beasting of various kinds.
a good related change would be to fix suggest the test only goal when a unit test fails. No idea where that message comes from though: reproduce with: ant test-only ...
I don't think this is a good idea, for the same reason you mentioned: skipping compilation. Yes, re-running a previously compiled test is the classic reproduction scenario, but a dev is IMHO just as likely to use your tweak-compile-test cycle in this case. Changing the printed repro line would serve as a form of advertisement for the feature generally, which I think is probably the wrong direction for this kind of functionality.