Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.1, 3.0.0-alpha1
    • Fix Version/s: 3.0.0-alpha1
    • Component/s: util
    • Labels:
      None
    • Environment:

      OS X, JDK 1.7.0_67, 1.8, Turkish locale

    • Hadoop Flags:
      Incompatible change
    • Release Note:
      When Hadoop JVMs create other processes on OS X, it will always use posix_spawn.

      Description

      OS X JDK has issues with localization that can cause util.Shell.run to fail. This is fixed in JDK9, but JDK7 and JDK8 are still broken.

      1. HADOOP-12495.00.patch
        0.6 kB
        Allen Wittenauer

        Activity

        Hide
        aw Allen Wittenauer added a comment -

        -00:

        • force the launchMechanism to be POSIX_SPAWN to avoid localization issues.
        Show
        aw Allen Wittenauer added a comment - -00: force the launchMechanism to be POSIX_SPAWN to avoid localization issues.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 11s docker + precommit patch detected.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 3m 21s trunk passed
        +1 compile 4m 49s trunk passed with JDK v1.8.0_60
        +1 compile 4m 32s trunk passed with JDK v1.7.0_79
        +1 checkstyle 0m 15s trunk passed
        +1 mvneclipse 0m 16s trunk passed
        +1 findbugs 1m 53s trunk passed
        +1 javadoc 1m 0s trunk passed with JDK v1.8.0_60
        +1 javadoc 1m 7s trunk passed with JDK v1.7.0_79
        +1 mvninstall 1m 27s the patch passed
        +1 compile 5m 2s the patch passed with JDK v1.8.0_60
        +1 javac 5m 2s the patch passed
        +1 compile 4m 55s the patch passed with JDK v1.7.0_79
        +1 javac 4m 55s the patch passed
        +1 checkstyle 0m 20s the patch passed
        +1 mvneclipse 0m 19s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 2m 13s the patch passed
        +1 javadoc 0m 56s the patch passed with JDK v1.8.0_60
        +1 javadoc 1m 10s the patch passed with JDK v1.7.0_79
        -1 unit 7m 41s hadoop-common in the patch failed with JDK v1.8.0_60.
        -1 unit 7m 38s hadoop-common in the patch failed with JDK v1.7.0_79.
        +1 asflicense 0m 24s Patch does not generate ASF License warnings.
        50m 37s



        Reason Tests
        JDK v1.7.0_79 Failed junit tests hadoop.fs.TestLocalFsFCStatistics
          hadoop.security.ssl.TestReloadingX509TrustManager
          hadoop.metrics2.impl.TestGangliaMetrics



        Subsystem Report/Notes
        Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-10-20
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12767659/HADOOP-12495.00.patch
        JIRA Issue HADOOP-12495
        Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile
        uname Linux cf0081e3ba94 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/patchprocess/apache-yetus-4ec64a8/dev-support/personality/hadoop.sh
        git revision trunk / 6c8b6f3
        Default Java 1.7.0_79
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/7895/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_60.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/7895/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_79.txt
        unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/7895/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/7895/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_79.txt
        JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7895/testReport/
        Max memory used 225MB
        Powered by Apache Yetus http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7895/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 3m 21s trunk passed +1 compile 4m 49s trunk passed with JDK v1.8.0_60 +1 compile 4m 32s trunk passed with JDK v1.7.0_79 +1 checkstyle 0m 15s trunk passed +1 mvneclipse 0m 16s trunk passed +1 findbugs 1m 53s trunk passed +1 javadoc 1m 0s trunk passed with JDK v1.8.0_60 +1 javadoc 1m 7s trunk passed with JDK v1.7.0_79 +1 mvninstall 1m 27s the patch passed +1 compile 5m 2s the patch passed with JDK v1.8.0_60 +1 javac 5m 2s the patch passed +1 compile 4m 55s the patch passed with JDK v1.7.0_79 +1 javac 4m 55s the patch passed +1 checkstyle 0m 20s the patch passed +1 mvneclipse 0m 19s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 13s the patch passed +1 javadoc 0m 56s the patch passed with JDK v1.8.0_60 +1 javadoc 1m 10s the patch passed with JDK v1.7.0_79 -1 unit 7m 41s hadoop-common in the patch failed with JDK v1.8.0_60. -1 unit 7m 38s hadoop-common in the patch failed with JDK v1.7.0_79. +1 asflicense 0m 24s Patch does not generate ASF License warnings. 50m 37s Reason Tests JDK v1.7.0_79 Failed junit tests hadoop.fs.TestLocalFsFCStatistics   hadoop.security.ssl.TestReloadingX509TrustManager   hadoop.metrics2.impl.TestGangliaMetrics Subsystem Report/Notes Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-10-20 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12767659/HADOOP-12495.00.patch JIRA Issue HADOOP-12495 Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile uname Linux cf0081e3ba94 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/patchprocess/apache-yetus-4ec64a8/dev-support/personality/hadoop.sh git revision trunk / 6c8b6f3 Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/7895/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/7895/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_79.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/7895/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/7895/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_79.txt JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7895/testReport/ Max memory used 225MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7895/console This message was automatically generated.
        Hide
        stevel@apache.org Steve Loughran added a comment -

        +1. Looks like it's turkey (something getting its case conversion wrong?). Hadoop is probably broken w.r.t turkey right now: to many .equalsIgnoreCase(), and toLower/toUpper that don't spec Locale.EN_US. Still, this a start

        Show
        stevel@apache.org Steve Loughran added a comment - +1. Looks like it's turkey (something getting its case conversion wrong?). Hadoop is probably broken w.r.t turkey right now: to many .equalsIgnoreCase(), and toLower/toUpper that don't spec Locale.EN_US. Still, this a start
        Hide
        aw Allen Wittenauer added a comment -

        I know all the bugs and such say it's the TR locale, but I don't have the Turkish locale installed on my machine and still see it. So I'm not sure if this is a new bug or what. lol

        Anyway, thanks for the review. I'll commit this so that at least this particular unit test failure goes away.

        Show
        aw Allen Wittenauer added a comment - I know all the bugs and such say it's the TR locale, but I don't have the Turkish locale installed on my machine and still see it. So I'm not sure if this is a new bug or what. lol Anyway, thanks for the review. I'll commit this so that at least this particular unit test failure goes away.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #8677 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8677/)
        HADOOP-12495. Fix posix_spawn error on OS X (aw) (aw: rev b37c41fd6ec704babf37325530e7cb9babf1ac0c)

        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8677 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8677/ ) HADOOP-12495 . Fix posix_spawn error on OS X (aw) (aw: rev b37c41fd6ec704babf37325530e7cb9babf1ac0c) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #577 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/577/)
        HADOOP-12495. Fix posix_spawn error on OS X (aw) (aw: rev b37c41fd6ec704babf37325530e7cb9babf1ac0c)

        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #577 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/577/ ) HADOOP-12495 . Fix posix_spawn error on OS X (aw) (aw: rev b37c41fd6ec704babf37325530e7cb9babf1ac0c) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #1298 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1298/)
        HADOOP-12495. Fix posix_spawn error on OS X (aw) (aw: rev b37c41fd6ec704babf37325530e7cb9babf1ac0c)

        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1298 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1298/ ) HADOOP-12495 . Fix posix_spawn error on OS X (aw) (aw: rev b37c41fd6ec704babf37325530e7cb9babf1ac0c) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #563 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/563/)
        HADOOP-12495. Fix posix_spawn error on OS X (aw) (aw: rev b37c41fd6ec704babf37325530e7cb9babf1ac0c)

        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #563 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/563/ ) HADOOP-12495 . Fix posix_spawn error on OS X (aw) (aw: rev b37c41fd6ec704babf37325530e7cb9babf1ac0c) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #2510 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2510/)
        HADOOP-12495. Fix posix_spawn error on OS X (aw) (aw: rev b37c41fd6ec704babf37325530e7cb9babf1ac0c)

        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2510 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2510/ ) HADOOP-12495 . Fix posix_spawn error on OS X (aw) (aw: rev b37c41fd6ec704babf37325530e7cb9babf1ac0c) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2459 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2459/)
        HADOOP-12495. Fix posix_spawn error on OS X (aw) (aw: rev b37c41fd6ec704babf37325530e7cb9babf1ac0c)

        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2459 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2459/ ) HADOOP-12495 . Fix posix_spawn error on OS X (aw) (aw: rev b37c41fd6ec704babf37325530e7cb9babf1ac0c) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #522 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/522/)
        HADOOP-12495. Fix posix_spawn error on OS X (aw) (aw: rev b37c41fd6ec704babf37325530e7cb9babf1ac0c)

        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #522 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/522/ ) HADOOP-12495 . Fix posix_spawn error on OS X (aw) (aw: rev b37c41fd6ec704babf37325530e7cb9babf1ac0c) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java

          People

          • Assignee:
            aw Allen Wittenauer
            Reporter:
            aw Allen Wittenauer
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development