Nigel, we already have a test in
MAPREDUCE-862 that validates output of a command like TestCLI does. The test captures the output and validates what's expected. However, the writer we use in the test case does not seem to require a flush to get the data. So, the test passes. However, when we tested on a real cluster, the flush was required with the System.out stream.
Also, it was not a simple addition to TestCLI. IIRC, we needed to add the queue command type to the TestCLI class. And that requires a patch to common. I think that is undesirable. TestCLI should be refactored so projects can add commands without having to submit patches to common, an effort for which there was no time close to the feature freeze.