XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.1.0
    • Component/s: build, core, examples, scripts, tests
    • Labels:
      None
    • Environment:

      Windows

      Description

      This is a large patch, please let me know if I should split it up and/or use the code reviewer tool. I tested it on linux and verified that it did not cause any unit test regressions.

      This patch fixes many unix-specific assumptions in code and tests:

      1. Using "/" as a path separator. On windows, the right separator is "\". Fix: create a new Path object with the directory and subdirectory names, which handles using the right separator.
      2. Determining whether a path is absolute by testing if it begins with "/". Fix: Create a new File object and use isAbsolute().
      3. Generating a pathname by concatenating "file://", getTestCaseDir(), File.separator, and the file name. This is wrong because this is a URI, and File.separator is "/" on unix but "\" on windows. Fix: New helper method getTestCaseFileUri solves this.
      4. Create directories by using "mkdir" shell command with hard-coded file separators. Fix: Create a new File object and use mkdirs().
      5. Scripts that use unix-only commands (ls, echo, etc.). Fix: Implement windows-specific commands and use Shell.WINDOWS to pick the right one at runtime.
      6. Assume that perl exists in TestShellActionExecutor.testPerlScript. It doesn't on Windows. Fix: skip the test on Windows.
      7. Use the test case name as part of the temp directory name in order to ensure directories are unique. This can cause issues on Windows. Fix: Use a random UUID instead.
      8. As a result of using getTestCaseUri, some test case verification that uses string equality fails. (TestLiteWorkflowAppService) Fix: use URI equivalence instead of string equality.
      9. AdminServlet tests verify that system variables are working by checking for a USER variable. This is called USERNAME in windows. Fix: On Windows, check for USERNAME instead.
      10. TestHostnameFilter checks that "localhost" is in the filter. On Windows, this is "127.0.0.1". Fix: On Windows, check for 127.0.0.1 instead.

      On windows there are still several tests failing. I hope to spend some time fixing these later:

      Failed tests:
      testActionCheck(org.apache.oozie.command.wf.TestActionCheckXCommand)
      testActionCheckTransientDuringLauncher(org.apache.oozie.command.wf.TestActionCheckXCommand)
      testActionCheckTransientDuringMRAction(org.apache.oozie.command.wf.TestActionCheckXCommand)
      testCoordinatorActionCommands(org.apache.oozie.sla.TestSLAEventGeneration): expected:<NOT_STARTED> but was:<IN_PROCESS>
      testChmodWithGlob(org.apache.oozie.action.hadoop.TestFsActionExecutor): expected:<rw[-]------> but was:<rw[x]------>
      Tests in error:
      testWaitFor(org.apache.oozie.test.TestXTestCase): could not delete path [D:\apache\oozie\core\target\test-data\oozietests\org.apache.oozie.test.TestXTestCase$MyXTestCase\testWaitFor\conf\oozie-site.xml]
      testBaseDir(org.apache.oozie.test.TestXTestCase): could not delete path [D:\apache\oozie\core\target\test-data\oozietests\org.apache.oozie.test.TestXTestCase$MyXTestCase\testBaseDir\conf\oozie-site.xml]
      

      How to run unit tests in windows:

      1. You will need to build hadoop-core from branch-1-win and install it into your local maven cache.
      2. In the root pom.xml, replace the dummy value in HADOOP_HOME with the path to the hadoop directory. The minicluster tests require bin\winutils.exe to be in this directory.
      3. In the root pom.xml (and hadooplibs version 1 pom.xmls, if OOZIE-1490 is not committed yet) change the hadoop version to the version built from branch-1-win.
      4. May also need to use Hive 0.11 with the windows fixes.

        Attachments

        1. OOZIE-1500.trunk.3.patch
          193 kB
          David Wannemacher
        2. OOZIE-1500.patch
          209 kB
          David Wannemacher
        3. OOZIE-1500.2.patch
          208 kB
          David Wannemacher

          Issue Links

            Activity

              People

              • Assignee:
                dwann David Wannemacher
                Reporter:
                dwann David Wannemacher
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: