In the course of investigation, updating current testConf.xml with ~140 tests to the new Hadoop smoke tests in testHDFSConf.xml with ~600 tests seemed to be proper.
It has been found that quite a few tests in testHDFSConf.xml, while passing on hadoop-common smoke infrastructure using a mini-cluster, fail on a default Bigtop cluster. There are a few unrelated reasons to that:
- The regexp for files in RegexpComparator in testHDFSConf.xml have a hard coded replication factor set to 1 that is fine for mini-cluster, but fails on a real cluster with a default replication factor set to 3. This issue should be resolved in HADOOP-9464.
- As opposed to Bigtop testConf.xml where default "root" working HDFS directory is set to /tmp/testcli, that is widely accessible, testHDFSConf.xml default HDFS directory is set to /. Since the "root" prefix needs to be prepended in each test case, a better solution would be to set HDFS / permission on a Bigtop cluster to 777.
- Output from test commands such as
are entirely different on mini-cluster and Bigtop cluster.
- Some tests require superuser privileges that needs to be addressed per test case.
These issues are planned to be resolved. Also, a reuse of the Hadoop testHDFSConf.xml along with the majority of the test helping infrastructure will be achieved by making TestCLI class extending CLITestHelperDFS as opposed to current CLITestHelper.