This is a large patch, please let me know if I should split it up and/or use the code reviewer tool. I tested it on linux and verified that it did not cause any unit test regressions.
This patch fixes many unix-specific assumptions in code and tests:
- Using "/" as a path separator. On windows, the right separator is "\". Fix: create a new Path object with the directory and subdirectory names, which handles using the right separator.
- Determining whether a path is absolute by testing if it begins with "/". Fix: Create a new File object and use isAbsolute().
- Generating a pathname by concatenating "file://", getTestCaseDir(), File.separator, and the file name. This is wrong because this is a URI, and File.separator is "/" on unix but "\" on windows. Fix: New helper method getTestCaseFileUri solves this.
- Create directories by using "mkdir" shell command with hard-coded file separators. Fix: Create a new File object and use mkdirs().
- Scripts that use unix-only commands (ls, echo, etc.). Fix: Implement windows-specific commands and use Shell.WINDOWS to pick the right one at runtime.
- Assume that perl exists in TestShellActionExecutor.testPerlScript. It doesn't on Windows. Fix: skip the test on Windows.
- Use the test case name as part of the temp directory name in order to ensure directories are unique. This can cause issues on Windows. Fix: Use a random UUID instead.
- As a result of using getTestCaseUri, some test case verification that uses string equality fails. (TestLiteWorkflowAppService) Fix: use URI equivalence instead of string equality.
- AdminServlet tests verify that system variables are working by checking for a USER variable. This is called USERNAME in windows. Fix: On Windows, check for USERNAME instead.
- TestHostnameFilter checks that "localhost" is in the filter. On Windows, this is "127.0.0.1". Fix: On Windows, check for 127.0.0.1 instead.
On windows there are still several tests failing. I hope to spend some time fixing these later:
How to run unit tests in windows:
- You will need to build hadoop-core from branch-1-win and install it into your local maven cache.
- In the root pom.xml, replace the dummy value in HADOOP_HOME with the path to the hadoop directory. The minicluster tests require bin\winutils.exe to be in this directory.
- In the root pom.xml (and hadooplibs version 1 pom.xmls, if
OOZIE-1490is not committed yet) change the hadoop version to the version built from branch-1-win.
- May also need to use Hive 0.11 with the windows fixes.