Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3029

FSDownload.unpack() uses local locale for FS case conversion, may not work everywhere

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.7.0
    • Component/s: nodemanager
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      FSDownload.unpack() lower-cases filenames in the local locale before looking at extensions for, "tar, "zip", ..

      String lowerDst = dst.getName().toLowerCase();
      

      it MUST use LOCALE_EN for the locale, else a file .ZIP won't be recognised as a zipfile in a turkish locale cluster

      1. YARN-3029-003.patch
        6 kB
        Tsuyoshi Ozawa
      2. YARN-3029.002.patch
        6 kB
        Varun Saxena
      3. YARN-3029.001.patch
        2 kB
        Varun Saxena

        Issue Links

          Activity

          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2040 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2040/)
          YARN-3029. FSDownload.unpack() uses local locale for FS case conversion, may not work everywhere. Contributed by Varun Saxena. (ozawa: rev 7acce7d3648d6f1e45ce280e2147e7dedf5693fc)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestFSDownload.java
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2040 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2040/ ) YARN-3029 . FSDownload.unpack() uses local locale for FS case conversion, may not work everywhere. Contributed by Varun Saxena. (ozawa: rev 7acce7d3648d6f1e45ce280e2147e7dedf5693fc) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestFSDownload.java hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #90 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/90/)
          YARN-3029. FSDownload.unpack() uses local locale for FS case conversion, may not work everywhere. Contributed by Varun Saxena. (ozawa: rev 7acce7d3648d6f1e45ce280e2147e7dedf5693fc)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestFSDownload.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #90 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/90/ ) YARN-3029 . FSDownload.unpack() uses local locale for FS case conversion, may not work everywhere. Contributed by Varun Saxena. (ozawa: rev 7acce7d3648d6f1e45ce280e2147e7dedf5693fc) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestFSDownload.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #86 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/86/)
          YARN-3029. FSDownload.unpack() uses local locale for FS case conversion, may not work everywhere. Contributed by Varun Saxena. (ozawa: rev 7acce7d3648d6f1e45ce280e2147e7dedf5693fc)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestFSDownload.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #86 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/86/ ) YARN-3029 . FSDownload.unpack() uses local locale for FS case conversion, may not work everywhere. Contributed by Varun Saxena. (ozawa: rev 7acce7d3648d6f1e45ce280e2147e7dedf5693fc) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestFSDownload.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2021 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2021/)
          YARN-3029. FSDownload.unpack() uses local locale for FS case conversion, may not work everywhere. Contributed by Varun Saxena. (ozawa: rev 7acce7d3648d6f1e45ce280e2147e7dedf5693fc)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestFSDownload.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2021 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2021/ ) YARN-3029 . FSDownload.unpack() uses local locale for FS case conversion, may not work everywhere. Contributed by Varun Saxena. (ozawa: rev 7acce7d3648d6f1e45ce280e2147e7dedf5693fc) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestFSDownload.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #823 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/823/)
          YARN-3029. FSDownload.unpack() uses local locale for FS case conversion, may not work everywhere. Contributed by Varun Saxena. (ozawa: rev 7acce7d3648d6f1e45ce280e2147e7dedf5693fc)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestFSDownload.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #823 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/823/ ) YARN-3029 . FSDownload.unpack() uses local locale for FS case conversion, may not work everywhere. Contributed by Varun Saxena. (ozawa: rev 7acce7d3648d6f1e45ce280e2147e7dedf5693fc) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestFSDownload.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #89 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/89/)
          YARN-3029. FSDownload.unpack() uses local locale for FS case conversion, may not work everywhere. Contributed by Varun Saxena. (ozawa: rev 7acce7d3648d6f1e45ce280e2147e7dedf5693fc)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestFSDownload.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #89 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/89/ ) YARN-3029 . FSDownload.unpack() uses local locale for FS case conversion, may not work everywhere. Contributed by Varun Saxena. (ozawa: rev 7acce7d3648d6f1e45ce280e2147e7dedf5693fc) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestFSDownload.java
          Hide
          varun_saxena Varun Saxena added a comment -

          Thanks Steve Loughran for review and reporting.
          Thanks Tsuyoshi Ozawa for the review and commit.

          Show
          varun_saxena Varun Saxena added a comment - Thanks Steve Loughran for review and reporting. Thanks Tsuyoshi Ozawa for the review and commit.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #6963 (See https://builds.apache.org/job/Hadoop-trunk-Commit/6963/)
          YARN-3029. FSDownload.unpack() uses local locale for FS case conversion, may not work everywhere. Contributed by Varun Saxena. (ozawa: rev 7acce7d3648d6f1e45ce280e2147e7dedf5693fc)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestFSDownload.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #6963 (See https://builds.apache.org/job/Hadoop-trunk-Commit/6963/ ) YARN-3029 . FSDownload.unpack() uses local locale for FS case conversion, may not work everywhere. Contributed by Varun Saxena. (ozawa: rev 7acce7d3648d6f1e45ce280e2147e7dedf5693fc) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestFSDownload.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java hadoop-yarn-project/CHANGES.txt
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          Committed this to trunk and branch-2. Thanks Varun for your contribution and thanks Steve for the review and sharing. I'll use the command next time.

          Show
          ozawa Tsuyoshi Ozawa added a comment - Committed this to trunk and branch-2. Thanks Varun for your contribution and thanks Steve for the review and sharing. I'll use the command next time.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          +1

          don't worry about trailing whitespace; when applying a patch go

          {{git apply --whitespace=fix ... }} and have it do the cleaning for you. Nobody is likely to complain that the patch applied differs from the final submission in terms of trailing whitespace.

          Show
          stevel@apache.org Steve Loughran added a comment - +1 don't worry about trailing whitespace; when applying a patch go {{git apply --whitespace=fix ... }} and have it do the cleaning for you. Nobody is likely to complain that the patch applied differs from the final submission in terms of trailing whitespace.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12695315/YARN-3029-003.patch
          against trunk revision 342efa1.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          -1 release audit. The applied patch generated 1 release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/6456//testReport/
          Release audit warnings: https://builds.apache.org/job/PreCommit-YARN-Build/6456//artifact/patchprocess/patchReleaseAuditProblems.txt
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6456//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12695315/YARN-3029-003.patch against trunk revision 342efa1. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. -1 release audit . The applied patch generated 1 release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/6456//testReport/ Release audit warnings: https://builds.apache.org/job/PreCommit-YARN-Build/6456//artifact/patchprocess/patchReleaseAuditProblems.txt Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6456//console This message is automatically generated.
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          Just removing trailing spaces from rev 002. The content has been reviewed already. Pending for Jenkins.

          Show
          ozawa Tsuyoshi Ozawa added a comment - Just removing trailing spaces from rev 002. The content has been reviewed already. Pending for Jenkins.
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          +1. There are some lines which include trailing spaces, but I'll remove them.

          Show
          ozawa Tsuyoshi Ozawa added a comment - +1. There are some lines which include trailing spaces, but I'll remove them.
          Hide
          hadoopqa Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12695106/YARN-3029.002.patch
          against trunk revision caf7298.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/6443//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6443//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12695106/YARN-3029.002.patch against trunk revision caf7298. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/6443//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6443//console This message is automatically generated.
          Hide
          varun_saxena Varun Saxena added a comment -

          Tsuyoshi Ozawa, added a test case. Kindly review.

          Show
          varun_saxena Varun Saxena added a comment - Tsuyoshi Ozawa , added a test case. Kindly review.
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          Varun Saxena can you add a test case?

          Show
          ozawa Tsuyoshi Ozawa added a comment - Varun Saxena can you add a test case?
          Hide
          varun_saxena Varun Saxena added a comment -

          Someone kindly review this patch.

          Show
          varun_saxena Varun Saxena added a comment - Someone kindly review this patch.
          Hide
          varun_saxena Varun Saxena added a comment -

          Steve Loughran, kindly review

          Show
          varun_saxena Varun Saxena added a comment - Steve Loughran , kindly review
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12691673/YARN-3029.001.patch
          against trunk revision 5b0d060.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/6309//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6309//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12691673/YARN-3029.001.patch against trunk revision 5b0d060. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/6309//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6309//console This message is automatically generated.
          Hide
          varun_saxena Varun Saxena added a comment -

          Attached a trivial patch

          Show
          varun_saxena Varun Saxena added a comment - Attached a trivial patch

            People

            • Assignee:
              varun_saxena Varun Saxena
              Reporter:
              stevel@apache.org Steve Loughran
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development