Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3982

container-executor parsing of container-executor.cfg broken in trunk and branch-2

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.8.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: nodemanager
    • Labels:
      None
    • Target Version/s:

      Description

      After YARN-2194, the container-executor parsing of container-executor.cfg is broken. The test-container-executor binary is also failing and has been failing for quite a while.

      1. YARN-3982.002.patch
        4 kB
        Varun Vasudev
      2. YARN-3982.001.patch
        4 kB
        Varun Vasudev

        Issue Links

          Activity

          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2216 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2216/)
          YARN-3982. container-executor parsing of container-executor.cfg broken (xgong: rev f17093421521efcbdc813f6f2b8411e45ecc7863)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/configuration.c
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2216 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2216/ ) YARN-3982 . container-executor parsing of container-executor.cfg broken (xgong: rev f17093421521efcbdc813f6f2b8411e45ecc7863) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/configuration.c hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #267 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/267/)
          YARN-3982. container-executor parsing of container-executor.cfg broken (xgong: rev f17093421521efcbdc813f6f2b8411e45ecc7863)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/configuration.c
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #267 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/267/ ) YARN-3982 . container-executor parsing of container-executor.cfg broken (xgong: rev f17093421521efcbdc813f6f2b8411e45ecc7863) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/configuration.c
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #259 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/259/)
          YARN-3982. container-executor parsing of container-executor.cfg broken (xgong: rev f17093421521efcbdc813f6f2b8411e45ecc7863)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/configuration.c
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #259 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/259/ ) YARN-3982 . container-executor parsing of container-executor.cfg broken (xgong: rev f17093421521efcbdc813f6f2b8411e45ecc7863) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/configuration.c hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2197 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2197/)
          YARN-3982. container-executor parsing of container-executor.cfg broken (xgong: rev f17093421521efcbdc813f6f2b8411e45ecc7863)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/configuration.c
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2197 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2197/ ) YARN-3982 . container-executor parsing of container-executor.cfg broken (xgong: rev f17093421521efcbdc813f6f2b8411e45ecc7863) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/configuration.c hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #1000 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1000/)
          YARN-3982. container-executor parsing of container-executor.cfg broken (xgong: rev f17093421521efcbdc813f6f2b8411e45ecc7863)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/configuration.c
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1000 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1000/ ) YARN-3982 . container-executor parsing of container-executor.cfg broken (xgong: rev f17093421521efcbdc813f6f2b8411e45ecc7863) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/configuration.c hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #270 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/270/)
          YARN-3982. container-executor parsing of container-executor.cfg broken (xgong: rev f17093421521efcbdc813f6f2b8411e45ecc7863)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/configuration.c
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #270 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/270/ ) YARN-3982 . container-executor parsing of container-executor.cfg broken (xgong: rev f17093421521efcbdc813f6f2b8411e45ecc7863) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/configuration.c hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8232 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8232/)
          YARN-3982. container-executor parsing of container-executor.cfg broken (xgong: rev f17093421521efcbdc813f6f2b8411e45ecc7863)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/configuration.c
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8232 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8232/ ) YARN-3982 . container-executor parsing of container-executor.cfg broken (xgong: rev f17093421521efcbdc813f6f2b8411e45ecc7863) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/configuration.c hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
          Hide
          xgong Xuan Gong added a comment -

          Committed into trunk/branch-2. Thanks, Varun.

          Show
          xgong Xuan Gong added a comment - Committed into trunk/branch-2. Thanks, Varun.
          Hide
          xgong Xuan Gong added a comment -

          +1 LGTM. Checking this in

          Show
          xgong Xuan Gong added a comment - +1 LGTM. Checking this in
          Hide
          hadoopqa Hadoop QA added a comment -



          +1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 5m 16s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 36s There were no new javac warning messages.
          +1 release audit 0m 20s The applied patch does not increase the total number of release audit warnings.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 18s mvn install still works.
          +1 eclipse:eclipse 0m 31s The patch built with eclipse:eclipse.
          +1 yarn tests 6m 8s Tests passed in hadoop-yarn-server-nodemanager.
              21m 12s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12747451/YARN-3982.002.patch
          Optional Tests javac unit
          git revision trunk / 3e6fce9
          hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8685/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8685/testReport/
          Java 1.7.0_55
          uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/8685/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 pre-patch 5m 16s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 36s There were no new javac warning messages. +1 release audit 0m 20s The applied patch does not increase the total number of release audit warnings. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 18s mvn install still works. +1 eclipse:eclipse 0m 31s The patch built with eclipse:eclipse. +1 yarn tests 6m 8s Tests passed in hadoop-yarn-server-nodemanager.     21m 12s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12747451/YARN-3982.002.patch Optional Tests javac unit git revision trunk / 3e6fce9 hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8685/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8685/testReport/ Java 1.7.0_55 uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/8685/console This message was automatically generated.
          Hide
          vvasudev Varun Vasudev added a comment -

          Which value delimiter are really used in here ? "%" or "," ?

          Sorry about the goof up in the comment. It should be ",". Uploaded a new patch with the fix.

          Show
          vvasudev Varun Vasudev added a comment - Which value delimiter are really used in here ? "%" or "," ? Sorry about the goof up in the comment. It should be ",". Uploaded a new patch with the fix.
          Hide
          xgong Xuan Gong added a comment -

          This doc is confusing.

          /**
           * Function to return an array of values for a key.
           * Value delimiter is assumed to be a '%'.
           */
          char ** get_values(const char * key) {
          

          Which value delimiter are really used in here ? "%" or "," ?

          Show
          xgong Xuan Gong added a comment - This doc is confusing. /** * Function to return an array of values for a key. * Value delimiter is assumed to be a '%'. */ char ** get_values( const char * key) { Which value delimiter are really used in here ? "%" or "," ?
          Hide
          vvasudev Varun Vasudev added a comment -

          Test failure is unrelated to the patch.

          Show
          vvasudev Varun Vasudev added a comment - Test failure is unrelated to the patch.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 5m 23s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 37s There were no new javac warning messages.
          +1 release audit 0m 20s The applied patch does not increase the total number of release audit warnings.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 22s mvn install still works.
          +1 eclipse:eclipse 0m 31s The patch built with eclipse:eclipse.
          -1 yarn tests 6m 5s Tests failed in hadoop-yarn-server-nodemanager.
              21m 21s  



          Reason Tests
          Failed unit tests hadoop.yarn.server.nodemanager.TestDeletionService



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12747417/YARN-3982.001.patch
          Optional Tests javac unit
          git revision trunk / 3e6fce9
          hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8683/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8683/testReport/
          Java 1.7.0_55
          uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/8683/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 5m 23s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 37s There were no new javac warning messages. +1 release audit 0m 20s The applied patch does not increase the total number of release audit warnings. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 22s mvn install still works. +1 eclipse:eclipse 0m 31s The patch built with eclipse:eclipse. -1 yarn tests 6m 5s Tests failed in hadoop-yarn-server-nodemanager.     21m 21s   Reason Tests Failed unit tests hadoop.yarn.server.nodemanager.TestDeletionService Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12747417/YARN-3982.001.patch Optional Tests javac unit git revision trunk / 3e6fce9 hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8683/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8683/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/8683/console This message was automatically generated.
          Hide
          vvasudev Varun Vasudev added a comment -

          Patch with fix attached.

          Show
          vvasudev Varun Vasudev added a comment - Patch with fix attached.

            People

            • Assignee:
              vvasudev Varun Vasudev
              Reporter:
              vvasudev Varun Vasudev
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development