Details

    • Type: Sub-task Sub-task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.4.3
    • Fix Version/s: 1.4.4
    • Component/s: None
    • Labels:
      None

      Description

      Some unit tests are failing when run on Windows. This happens mainly for File System difference like File path format, file extension differences (e.g. scripts have extension .cmd on windows as opposed to .sh), .. etc

      1. SQOOP-955.trunk.3.patch
        10 kB
        Ahmed El Baz
      2. SQOOP-955.trunk.1.patch
        11 kB
        Ahmed El Baz

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Sqoop-ant-jdk-1.6-hadoop100 #628 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/628/)
          SQOOP-955: Fix Sqoop unit test failures on Windows (Revision f5006b60059dda44e6b77e6f29749e2fbcab5bf1)

          Result = SUCCESS
          jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=f5006b60059dda44e6b77e6f29749e2fbcab5bf1
          Files :

          • testdata/hive/bin/hive.cmd
          • src/test/com/cloudera/sqoop/lib/TestBlobRef.java
          • src/test/com/cloudera/sqoop/io/TestNamedFifo.java
          • src/test/com/cloudera/sqoop/hive/TestHiveImport.java
          • src/java/org/apache/sqoop/util/ClassLoaderStack.java
          • src/test/com/cloudera/sqoop/orm/TestClassWriter.java
          • src/test/com/cloudera/sqoop/lib/TestClobRef.java
          • src/java/org/apache/sqoop/orm/CompilationManager.java
          • src/java/org/apache/sqoop/hive/HiveImport.java
          Show
          Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop100 #628 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/628/ ) SQOOP-955 : Fix Sqoop unit test failures on Windows (Revision f5006b60059dda44e6b77e6f29749e2fbcab5bf1) Result = SUCCESS jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=f5006b60059dda44e6b77e6f29749e2fbcab5bf1 Files : testdata/hive/bin/hive.cmd src/test/com/cloudera/sqoop/lib/TestBlobRef.java src/test/com/cloudera/sqoop/io/TestNamedFifo.java src/test/com/cloudera/sqoop/hive/TestHiveImport.java src/java/org/apache/sqoop/util/ClassLoaderStack.java src/test/com/cloudera/sqoop/orm/TestClassWriter.java src/test/com/cloudera/sqoop/lib/TestClobRef.java src/java/org/apache/sqoop/orm/CompilationManager.java src/java/org/apache/sqoop/hive/HiveImport.java
          Hide
          Hudson added a comment -

          Integrated in Sqoop-ant-jdk-1.6-hadoop200 #620 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/620/)
          SQOOP-955: Fix Sqoop unit test failures on Windows (Revision f5006b60059dda44e6b77e6f29749e2fbcab5bf1)

          Result = SUCCESS
          jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=f5006b60059dda44e6b77e6f29749e2fbcab5bf1
          Files :

          • src/test/com/cloudera/sqoop/hive/TestHiveImport.java
          • src/java/org/apache/sqoop/hive/HiveImport.java
          • src/test/com/cloudera/sqoop/orm/TestClassWriter.java
          • src/test/com/cloudera/sqoop/lib/TestClobRef.java
          • src/test/com/cloudera/sqoop/io/TestNamedFifo.java
          • src/java/org/apache/sqoop/util/ClassLoaderStack.java
          • src/test/com/cloudera/sqoop/lib/TestBlobRef.java
          • src/java/org/apache/sqoop/orm/CompilationManager.java
          • testdata/hive/bin/hive.cmd
          Show
          Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop200 #620 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/620/ ) SQOOP-955 : Fix Sqoop unit test failures on Windows (Revision f5006b60059dda44e6b77e6f29749e2fbcab5bf1) Result = SUCCESS jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=f5006b60059dda44e6b77e6f29749e2fbcab5bf1 Files : src/test/com/cloudera/sqoop/hive/TestHiveImport.java src/java/org/apache/sqoop/hive/HiveImport.java src/test/com/cloudera/sqoop/orm/TestClassWriter.java src/test/com/cloudera/sqoop/lib/TestClobRef.java src/test/com/cloudera/sqoop/io/TestNamedFifo.java src/java/org/apache/sqoop/util/ClassLoaderStack.java src/test/com/cloudera/sqoop/lib/TestBlobRef.java src/java/org/apache/sqoop/orm/CompilationManager.java testdata/hive/bin/hive.cmd
          Hide
          Hudson added a comment -

          Integrated in Sqoop-ant-jdk-1.6-hadoop23 #828 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/828/)
          SQOOP-955: Fix Sqoop unit test failures on Windows (Revision f5006b60059dda44e6b77e6f29749e2fbcab5bf1)

          Result = SUCCESS
          jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=f5006b60059dda44e6b77e6f29749e2fbcab5bf1
          Files :

          • src/test/com/cloudera/sqoop/hive/TestHiveImport.java
          • src/java/org/apache/sqoop/hive/HiveImport.java
          • testdata/hive/bin/hive.cmd
          • src/test/com/cloudera/sqoop/io/TestNamedFifo.java
          • src/java/org/apache/sqoop/orm/CompilationManager.java
          • src/test/com/cloudera/sqoop/lib/TestClobRef.java
          • src/test/com/cloudera/sqoop/lib/TestBlobRef.java
          • src/java/org/apache/sqoop/util/ClassLoaderStack.java
          • src/test/com/cloudera/sqoop/orm/TestClassWriter.java
          Show
          Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop23 #828 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/828/ ) SQOOP-955 : Fix Sqoop unit test failures on Windows (Revision f5006b60059dda44e6b77e6f29749e2fbcab5bf1) Result = SUCCESS jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=f5006b60059dda44e6b77e6f29749e2fbcab5bf1 Files : src/test/com/cloudera/sqoop/hive/TestHiveImport.java src/java/org/apache/sqoop/hive/HiveImport.java testdata/hive/bin/hive.cmd src/test/com/cloudera/sqoop/io/TestNamedFifo.java src/java/org/apache/sqoop/orm/CompilationManager.java src/test/com/cloudera/sqoop/lib/TestClobRef.java src/test/com/cloudera/sqoop/lib/TestBlobRef.java src/java/org/apache/sqoop/util/ClassLoaderStack.java src/test/com/cloudera/sqoop/orm/TestClassWriter.java
          Hide
          Hudson added a comment -

          Integrated in Sqoop-ant-jdk-1.6-hadoop20 #620 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/620/)
          SQOOP-955: Fix Sqoop unit test failures on Windows (Revision f5006b60059dda44e6b77e6f29749e2fbcab5bf1)

          Result = SUCCESS
          jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=f5006b60059dda44e6b77e6f29749e2fbcab5bf1
          Files :

          • src/java/org/apache/sqoop/hive/HiveImport.java
          • testdata/hive/bin/hive.cmd
          • src/test/com/cloudera/sqoop/io/TestNamedFifo.java
          • src/java/org/apache/sqoop/util/ClassLoaderStack.java
          • src/test/com/cloudera/sqoop/lib/TestClobRef.java
          • src/test/com/cloudera/sqoop/lib/TestBlobRef.java
          • src/test/com/cloudera/sqoop/hive/TestHiveImport.java
          • src/test/com/cloudera/sqoop/orm/TestClassWriter.java
          • src/java/org/apache/sqoop/orm/CompilationManager.java
          Show
          Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop20 #620 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/620/ ) SQOOP-955 : Fix Sqoop unit test failures on Windows (Revision f5006b60059dda44e6b77e6f29749e2fbcab5bf1) Result = SUCCESS jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=f5006b60059dda44e6b77e6f29749e2fbcab5bf1 Files : src/java/org/apache/sqoop/hive/HiveImport.java testdata/hive/bin/hive.cmd src/test/com/cloudera/sqoop/io/TestNamedFifo.java src/java/org/apache/sqoop/util/ClassLoaderStack.java src/test/com/cloudera/sqoop/lib/TestClobRef.java src/test/com/cloudera/sqoop/lib/TestBlobRef.java src/test/com/cloudera/sqoop/hive/TestHiveImport.java src/test/com/cloudera/sqoop/orm/TestClassWriter.java src/java/org/apache/sqoop/orm/CompilationManager.java
          Show
          Jarek Jarcec Cecho added a comment - The patch is in: https://git-wip-us.apache.org/repos/asf?p=sqoop.git;a=commit;h=f5006b60059dda44e6b77e6f29749e2fbcab5bf1 Thank you Ahmed! Jarcec
          Hide
          Ahmed El Baz added a comment -

          Updated the patch based on comments and discussions in the review board.
          Thanks,

          Ahmed

          Show
          Ahmed El Baz added a comment - Updated the patch based on comments and discussions in the review board. Thanks, Ahmed
          Hide
          Ahmed El Baz added a comment -

          Attaching a patch to fix Unit Test failures on Windows.

          Change details:

          Provide a Windows version for hive.cmd under testdata/hive/bin

          Construct paths using File and URL classes rather than string formatting. This takes care of abstracting paths to be OS independent.

          Ensure that classpath uses the URL separator '/' rather than the default File seprator '\' on Windows.

          Invoke the correct script (hive.cmd vs hive) based on the host OS in HiveImport

          Change TestClobRef and TestBlobRef to use "true" for recursive delete, otherwise the directory delete fails on Windows with a "Directory not empty" message.

          Note:
          This change requires a Hadoop version supporting Windows. One way is to have it installed in local cache, and let Sqoop use it as follows:

          ant clean test -Dhadoopversion=100 -Dhadoop.version=1.1.0-SNAPSHOT -Dresolvers=internal

          I will make sure this is added into the documentation as part of the Documentation Sub task SQOOP-960

          Show
          Ahmed El Baz added a comment - Attaching a patch to fix Unit Test failures on Windows. Change details: Provide a Windows version for hive.cmd under testdata/hive/bin Construct paths using File and URL classes rather than string formatting. This takes care of abstracting paths to be OS independent. Ensure that classpath uses the URL separator '/' rather than the default File seprator '\' on Windows. Invoke the correct script (hive.cmd vs hive) based on the host OS in HiveImport Change TestClobRef and TestBlobRef to use "true" for recursive delete, otherwise the directory delete fails on Windows with a "Directory not empty" message. Note: This change requires a Hadoop version supporting Windows. One way is to have it installed in local cache, and let Sqoop use it as follows: ant clean test -Dhadoopversion=100 -Dhadoop.version=1.1.0-SNAPSHOT -Dresolvers=internal I will make sure this is added into the documentation as part of the Documentation Sub task SQOOP-960

            People

            • Assignee:
              Ahmed El Baz
              Reporter:
              Ahmed El Baz
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development