Thanks for such detailed feedbacks Masatake Iwasaki, it helped a lot!
Given what you mentioned here, I did some more investigation, and I think I verified that it is possible for one test to read config created by another.
The two classes both have this line:
Which writes the config to the exactly same file "hadoop-metrics2-test.properties" first. Then both classes have the call:
Which reads the file. So there can be race condition from the write to the read. This error can be reproduced by adding delay between .save() and ms.start() call, and run the tests at the same time.
Another problem with this race condition is the missing metric error of checkMetrics() as mentioned in
HADOOP-12588. For example, one class might have the following in its config
which configures to remove Xxx from the metric sources. If the other class is expecting this metric, we would end up getting this error:
java.lang.AssertionError: Missing metrics: test.s1rec.Xxx
One solution I'm thinking of is to separate the config file they are writing to by introducing a unique prefix for each test case, such that each test uses a different config file, similar to RollingFileSystemSinkTestBase#initMetricsSystem. The pros is that we are still able to run test cases in parallel without race condition, and the cons is that there might be a couple tens more .properties generated. What do you think about this approach? Or do you have any other suggestions?