Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha2
    • Component/s: scheduler
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    1. YARN-4506.1.patch
      34 kB
      Rohith Sharma K S
    2. YARN-4206.6.patch
      38 kB
      Rohith Sharma K S
    3. YARN-4206.5.patch
      40 kB
      Rohith Sharma K S
    4. YARN-4206.4.patch
      38 kB
      Rohith Sharma K S
    5. YARN-4206.3.patch
      38 kB
      Rohith Sharma K S
    6. YARN-4206.2.patch
      39 kB
      Rohith Sharma K S

      Issue Links

        Activity

        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10887 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10887/)
        YARN-4206. Add Application timeouts in Application report and CLI. (sunil: rev eb0a483ed07399c7a06ed9db4bfbec382de470da)

        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestApplicationLifetimeMonitor.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/YarnClient.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationReport.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/impl/YarnClientImpl.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/ApplicationCLI.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationReportPBImpl.java
        • (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationTimeout.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
        • (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationTimeoutPBImpl.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/TestPBImplRecords.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10887 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10887/ ) YARN-4206 . Add Application timeouts in Application report and CLI. (sunil: rev eb0a483ed07399c7a06ed9db4bfbec382de470da) (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestApplicationLifetimeMonitor.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/YarnClient.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationReport.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/impl/YarnClientImpl.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/ApplicationCLI.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationReportPBImpl.java (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationTimeout.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationTimeoutPBImpl.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/TestPBImplRecords.java
        Hide
        sunilg Sunil G added a comment -

        Thanks Rohith Sharma K S for the patch. Thanks Jian He Vinod Kumar Vavilapalli Gour Saha for additional reviews!

        Show
        sunilg Sunil G added a comment - Thanks Rohith Sharma K S for the patch. Thanks Jian He Vinod Kumar Vavilapalli Gour Saha for additional reviews!
        Hide
        sunilg Sunil G added a comment -

        YARN-5920 would have fixed by TestRMHA failure. However it seems this jenkins run was not having that fix.

        +1 to latest patch. Committing shortly..

        Show
        sunilg Sunil G added a comment - YARN-5920 would have fixed by TestRMHA failure. However it seems this jenkins run was not having that fix. +1 to latest patch. Committing shortly..
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 30s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
        0 mvndep 0m 53s Maven dependency ordering for branch
        +1 mvninstall 8m 16s trunk passed
        +1 compile 5m 5s trunk passed
        +1 checkstyle 0m 50s trunk passed
        +1 mvnsite 2m 26s trunk passed
        +1 mvneclipse 1m 19s trunk passed
        +1 findbugs 4m 3s trunk passed
        +1 javadoc 1m 46s trunk passed
        0 mvndep 0m 11s Maven dependency ordering for patch
        +1 mvninstall 1m 49s the patch passed
        +1 compile 4m 51s the patch passed
        +1 cc 4m 51s the patch passed
        +1 javac 4m 51s the patch passed
        -0 checkstyle 0m 48s hadoop-yarn-project/hadoop-yarn: The patch generated 5 new + 374 unchanged - 2 fixed = 379 total (was 376)
        +1 mvnsite 2m 51s the patch passed
        +1 mvneclipse 1m 28s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 5m 7s the patch passed
        +1 javadoc 2m 8s the patch passed
        +1 unit 0m 40s hadoop-yarn-api in the patch passed.
        +1 unit 3m 3s hadoop-yarn-common in the patch passed.
        -1 unit 50m 34s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 unit 17m 20s hadoop-yarn-client in the patch passed.
        +1 asflicense 0m 31s The patch does not generate ASF License warnings.
        125m 15s



        Reason Tests
        Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart
          hadoop.yarn.server.resourcemanager.TestRMHA



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue YARN-4206
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840375/YARN-4206.6.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux f33fc337371a 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / c7a5f29
        Default Java 1.8.0_111
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/14071/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/14071/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14071/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/14071/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 30s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 53s Maven dependency ordering for branch +1 mvninstall 8m 16s trunk passed +1 compile 5m 5s trunk passed +1 checkstyle 0m 50s trunk passed +1 mvnsite 2m 26s trunk passed +1 mvneclipse 1m 19s trunk passed +1 findbugs 4m 3s trunk passed +1 javadoc 1m 46s trunk passed 0 mvndep 0m 11s Maven dependency ordering for patch +1 mvninstall 1m 49s the patch passed +1 compile 4m 51s the patch passed +1 cc 4m 51s the patch passed +1 javac 4m 51s the patch passed -0 checkstyle 0m 48s hadoop-yarn-project/hadoop-yarn: The patch generated 5 new + 374 unchanged - 2 fixed = 379 total (was 376) +1 mvnsite 2m 51s the patch passed +1 mvneclipse 1m 28s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 5m 7s the patch passed +1 javadoc 2m 8s the patch passed +1 unit 0m 40s hadoop-yarn-api in the patch passed. +1 unit 3m 3s hadoop-yarn-common in the patch passed. -1 unit 50m 34s hadoop-yarn-server-resourcemanager in the patch failed. +1 unit 17m 20s hadoop-yarn-client in the patch passed. +1 asflicense 0m 31s The patch does not generate ASF License warnings. 125m 15s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart   hadoop.yarn.server.resourcemanager.TestRMHA Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-4206 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840375/YARN-4206.6.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux f33fc337371a 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / c7a5f29 Default Java 1.8.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/14071/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/14071/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14071/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/14071/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        jianhe Jian He added a comment -

        +1

        Show
        jianhe Jian He added a comment - +1
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        After having discussion with Jian and other folks, they feel that API should be more clarity and easy enough to input values. Providing multiple options makes user to bit confusing state. So, going ahed with 2nd suggestion i.e ./yarn application -appId $appId -updateLifetime $timeout.
        Attached the patch reflect the same.

        Show
        rohithsharma Rohith Sharma K S added a comment - After having discussion with Jian and other folks, they feel that API should be more clarity and easy enough to input values. Providing multiple options makes user to bit confusing state. So, going ahed with 2nd suggestion i.e ./yarn application -appId $appId -updateLifetime $timeout. Attached the patch reflect the same.
        Hide
        jianhe Jian He added a comment -

        would a single option -updateLifetime be simper ? having one more option makes user memorize one more option...

        Show
        jianhe Jian He added a comment - would a single option -updateLifetime be simper ? having one more option makes user memorize one more option...
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 13s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
        0 mvndep 0m 11s Maven dependency ordering for branch
        +1 mvninstall 6m 52s trunk passed
        +1 compile 5m 5s trunk passed
        +1 checkstyle 0m 49s trunk passed
        +1 mvnsite 2m 22s trunk passed
        +1 mvneclipse 1m 20s trunk passed
        +1 findbugs 3m 59s trunk passed
        +1 javadoc 1m 49s trunk passed
        0 mvndep 0m 11s Maven dependency ordering for patch
        +1 mvninstall 1m 53s the patch passed
        +1 compile 5m 0s the patch passed
        +1 cc 5m 0s the patch passed
        +1 javac 5m 0s the patch passed
        -0 checkstyle 0m 50s hadoop-yarn-project/hadoop-yarn: The patch generated 11 new + 374 unchanged - 2 fixed = 385 total (was 376)
        +1 mvnsite 2m 31s the patch passed
        +1 mvneclipse 1m 18s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 4m 32s the patch passed
        +1 javadoc 1m 42s the patch passed
        +1 unit 0m 33s hadoop-yarn-api in the patch passed.
        +1 unit 2m 27s hadoop-yarn-common in the patch passed.
        +1 unit 38m 42s hadoop-yarn-server-resourcemanager in the patch passed.
        +1 unit 16m 17s hadoop-yarn-client in the patch passed.
        +1 asflicense 0m 29s The patch does not generate ASF License warnings.
        107m 40s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue YARN-4206
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840247/YARN-4206.5.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux 831d89e5b8d2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 3541ed8
        Default Java 1.8.0_111
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/14052/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14052/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/14052/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 11s Maven dependency ordering for branch +1 mvninstall 6m 52s trunk passed +1 compile 5m 5s trunk passed +1 checkstyle 0m 49s trunk passed +1 mvnsite 2m 22s trunk passed +1 mvneclipse 1m 20s trunk passed +1 findbugs 3m 59s trunk passed +1 javadoc 1m 49s trunk passed 0 mvndep 0m 11s Maven dependency ordering for patch +1 mvninstall 1m 53s the patch passed +1 compile 5m 0s the patch passed +1 cc 5m 0s the patch passed +1 javac 5m 0s the patch passed -0 checkstyle 0m 50s hadoop-yarn-project/hadoop-yarn: The patch generated 11 new + 374 unchanged - 2 fixed = 385 total (was 376) +1 mvnsite 2m 31s the patch passed +1 mvneclipse 1m 18s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 4m 32s the patch passed +1 javadoc 1m 42s the patch passed +1 unit 0m 33s hadoop-yarn-api in the patch passed. +1 unit 2m 27s hadoop-yarn-common in the patch passed. +1 unit 38m 42s hadoop-yarn-server-resourcemanager in the patch passed. +1 unit 16m 17s hadoop-yarn-client in the patch passed. +1 asflicense 0m 29s The patch does not generate ASF License warnings. 107m 40s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-4206 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840247/YARN-4206.5.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 831d89e5b8d2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 3541ed8 Default Java 1.8.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/14052/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14052/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/14052/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        Updated patch fixing Vinod review comments.

        Show
        rohithsharma Rohith Sharma K S added a comment - Updated patch fixing Vinod review comments.
        Hide
        vinodkv Vinod Kumar Vavilapalli added a comment -

        Few comments:

        • Rename suggestions
          • getApplicationTimeoutList -> getApplicationTimeouts & setApplicationTimeoutList -> setApplicationTimeouts
          • getAppTimeoutType -> getTimeoutType & setAppTimeoutType -> setTimeoutType
          • getExpireTime -> getExpiryTime
        • Drop the "of an application" part in the javadoc everywhere to keep it open for app, app-attempt, container timeouts? For e.g. in "Get <code>expireTime</code> of an application for given timeout type."
        • From the server side, we should always return a timeout object - for now only with LIFETime type.
          • If an app didn't specify anything: expiry-time = "unlimited", remaining-time = -1 (unknown)
          • If an app specified the timeout and is running, return proper values
          • If an app is finished, expiry-time = <proper value | unlimited>, remaining-time = 0
        • CLI
          • I think we should either implement (a) your earlier proposal (./yarn application -updateTimeout 10 -updateType LIFETIME) as it is foolproof., then madate updateType with updateTimeout and for now only support LIFETIME as a type in the CLI or (b) have a simpler updateLifetime.
          • +        ApplicationTimeout timeout =
            +            appReport.getApplicationTimeoutList().iterator().next();
            +        appReportStr.print("\tApplication Expire Time : ");
            +        appReportStr.println(timeout.getExpireTime());
            +        appReportStr.print("\tApplication Remaining Time : ");
            +        appReportStr.print(timeout.getRemainingTime());
            
          • We should simply iterate the timeouts and print one line for each type: "Lifetime: expiry-time=<ISO format>, remaining-time=<long>seconds"
        Show
        vinodkv Vinod Kumar Vavilapalli added a comment - Few comments: Rename suggestions getApplicationTimeoutList -> getApplicationTimeouts & setApplicationTimeoutList -> setApplicationTimeouts getAppTimeoutType -> getTimeoutType & setAppTimeoutType -> setTimeoutType getExpireTime -> getExpiryTime Drop the "of an application" part in the javadoc everywhere to keep it open for app, app-attempt, container timeouts? For e.g. in "Get <code>expireTime</code> of an application for given timeout type." From the server side, we should always return a timeout object - for now only with LIFETime type. If an app didn't specify anything: expiry-time = "unlimited", remaining-time = -1 (unknown) If an app specified the timeout and is running, return proper values If an app is finished, expiry-time = <proper value | unlimited>, remaining-time = 0 CLI I think we should either implement (a) your earlier proposal ( ./yarn application -updateTimeout 10 -updateType LIFETIME ) as it is foolproof., then madate updateType with updateTimeout and for now only support LIFETIME as a type in the CLI or (b) have a simpler updateLifetime. + ApplicationTimeout timeout = + appReport.getApplicationTimeoutList().iterator().next(); + appReportStr.print( "\tApplication Expire Time : " ); + appReportStr.println(timeout.getExpireTime()); + appReportStr.print( "\tApplication Remaining Time : " ); + appReportStr.print(timeout.getRemainingTime()); We should simply iterate the timeouts and print one line for each type: "Lifetime: expiry-time=<ISO format>, remaining-time=<long>seconds"
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 18s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
        0 mvndep 0m 54s Maven dependency ordering for branch
        +1 mvninstall 6m 50s trunk passed
        +1 compile 4m 59s trunk passed
        +1 checkstyle 0m 48s trunk passed
        +1 mvnsite 2m 21s trunk passed
        +1 mvneclipse 1m 19s trunk passed
        +1 findbugs 3m 49s trunk passed
        +1 javadoc 1m 44s trunk passed
        0 mvndep 0m 11s Maven dependency ordering for patch
        +1 mvninstall 1m 42s the patch passed
        +1 compile 4m 38s the patch passed
        +1 cc 4m 38s the patch passed
        +1 javac 4m 38s the patch passed
        -0 checkstyle 0m 48s hadoop-yarn-project/hadoop-yarn: The patch generated 5 new + 374 unchanged - 2 fixed = 379 total (was 376)
        +1 mvnsite 2m 20s the patch passed
        +1 mvneclipse 1m 17s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 4m 17s the patch passed
        +1 javadoc 1m 40s the patch passed
        +1 unit 0m 32s hadoop-yarn-api in the patch passed.
        +1 unit 2m 26s hadoop-yarn-common in the patch passed.
        -1 unit 42m 5s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 unit 16m 24s hadoop-yarn-client in the patch passed.
        +1 asflicense 0m 29s The patch does not generate ASF License warnings.
        110m 31s



        Reason Tests
        Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue YARN-4206
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840198/YARN-4206.4.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux 8a217f838e7d 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 4667564
        Default Java 1.8.0_111
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/14048/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/14048/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14048/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/14048/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 54s Maven dependency ordering for branch +1 mvninstall 6m 50s trunk passed +1 compile 4m 59s trunk passed +1 checkstyle 0m 48s trunk passed +1 mvnsite 2m 21s trunk passed +1 mvneclipse 1m 19s trunk passed +1 findbugs 3m 49s trunk passed +1 javadoc 1m 44s trunk passed 0 mvndep 0m 11s Maven dependency ordering for patch +1 mvninstall 1m 42s the patch passed +1 compile 4m 38s the patch passed +1 cc 4m 38s the patch passed +1 javac 4m 38s the patch passed -0 checkstyle 0m 48s hadoop-yarn-project/hadoop-yarn: The patch generated 5 new + 374 unchanged - 2 fixed = 379 total (was 376) +1 mvnsite 2m 20s the patch passed +1 mvneclipse 1m 17s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 4m 17s the patch passed +1 javadoc 1m 40s the patch passed +1 unit 0m 32s hadoop-yarn-api in the patch passed. +1 unit 2m 26s hadoop-yarn-common in the patch passed. -1 unit 42m 5s hadoop-yarn-server-resourcemanager in the patch failed. +1 unit 16m 24s hadoop-yarn-client in the patch passed. +1 asflicense 0m 29s The patch does not generate ASF License warnings. 110m 31s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-4206 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840198/YARN-4206.4.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 8a217f838e7d 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 4667564 Default Java 1.8.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/14048/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/14048/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14048/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/14048/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        updated patch fixing test failure.

        Show
        rohithsharma Rohith Sharma K S added a comment - updated patch fixing test failure.
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        It is related to patch, let me upload a new patch with test case correction.

        Show
        rohithsharma Rohith Sharma K S added a comment - It is related to patch, let me upload a new patch with test case correction.
        Hide
        jianhe Jian He added a comment -

        UT failed, Rohith Sharma K S, could you take a look ?

        Show
        jianhe Jian He added a comment - UT failed, Rohith Sharma K S , could you take a look ?
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 15s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
        0 mvndep 0m 10s Maven dependency ordering for branch
        +1 mvninstall 8m 8s trunk passed
        +1 compile 5m 30s trunk passed
        +1 checkstyle 0m 49s trunk passed
        +1 mvnsite 2m 32s trunk passed
        +1 mvneclipse 1m 21s trunk passed
        +1 findbugs 4m 2s trunk passed
        +1 javadoc 1m 44s trunk passed
        0 mvndep 0m 9s Maven dependency ordering for patch
        +1 mvninstall 1m 41s the patch passed
        +1 compile 4m 39s the patch passed
        +1 cc 4m 39s the patch passed
        +1 javac 4m 39s the patch passed
        -0 checkstyle 0m 48s hadoop-yarn-project/hadoop-yarn: The patch generated 8 new + 374 unchanged - 2 fixed = 382 total (was 376)
        +1 mvnsite 2m 22s the patch passed
        +1 mvneclipse 1m 21s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 5m 4s the patch passed
        +1 javadoc 1m 46s the patch passed
        +1 unit 0m 32s hadoop-yarn-api in the patch passed.
        +1 unit 2m 37s hadoop-yarn-common in the patch passed.
        +1 unit 38m 58s hadoop-yarn-server-resourcemanager in the patch passed.
        -1 unit 16m 33s hadoop-yarn-client in the patch failed.
        +1 asflicense 0m 34s The patch does not generate ASF License warnings.
        110m 15s



        Reason Tests
        Failed junit tests hadoop.yarn.client.cli.TestYarnCLI



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue YARN-4206
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840015/YARN-4206.3.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux ec1041d4d77d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 613b902
        Default Java 1.8.0_111
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/14026/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/14026/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14026/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/14026/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 10s Maven dependency ordering for branch +1 mvninstall 8m 8s trunk passed +1 compile 5m 30s trunk passed +1 checkstyle 0m 49s trunk passed +1 mvnsite 2m 32s trunk passed +1 mvneclipse 1m 21s trunk passed +1 findbugs 4m 2s trunk passed +1 javadoc 1m 44s trunk passed 0 mvndep 0m 9s Maven dependency ordering for patch +1 mvninstall 1m 41s the patch passed +1 compile 4m 39s the patch passed +1 cc 4m 39s the patch passed +1 javac 4m 39s the patch passed -0 checkstyle 0m 48s hadoop-yarn-project/hadoop-yarn: The patch generated 8 new + 374 unchanged - 2 fixed = 382 total (was 376) +1 mvnsite 2m 22s the patch passed +1 mvneclipse 1m 21s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 5m 4s the patch passed +1 javadoc 1m 46s the patch passed +1 unit 0m 32s hadoop-yarn-api in the patch passed. +1 unit 2m 37s hadoop-yarn-common in the patch passed. +1 unit 38m 58s hadoop-yarn-server-resourcemanager in the patch passed. -1 unit 16m 33s hadoop-yarn-client in the patch failed. +1 asflicense 0m 34s The patch does not generate ASF License warnings. 110m 15s Reason Tests Failed junit tests hadoop.yarn.client.cli.TestYarnCLI Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-4206 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840015/YARN-4206.3.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux ec1041d4d77d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 613b902 Default Java 1.8.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/14026/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/14026/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14026/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/14026/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        jianhe Jian He added a comment -

        +1

        Show
        jianhe Jian He added a comment - +1
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        updated patch with following changes

        1. fixed Jian review comment i.e log message change
        2. Removed support for sub option i.e -updateType since only one timeout type is supported. In future if any plan to support new timeout, then LIFETIME will become default value. New CLI API remain as ./yarn application -appId <appId> -updateTimeout <Timeout>
        Show
        rohithsharma Rohith Sharma K S added a comment - updated patch with following changes fixed Jian review comment i.e log message change Removed support for sub option i.e -updateType since only one timeout type is supported. In future if any plan to support new timeout, then LIFETIME will become default value. New CLI API remain as ./yarn application -appId <appId> -updateTimeout <Timeout>
        Hide
        jianhe Jian He added a comment -

        minor comment on the logging

         sysout.println("Updating timeout of an application " + applicationId);
        sysout.println("Successfully updated timeout of an application "
                + applicationId + ". New expire time will be " + newTimeout);
        

        may be explicitly mention the timeoutType ? like:

         sysout.println("Updating " + timeoutType + " of an application " + applicationId);
        sysout.println("Successfully updated " + timeoutType +" of an application "
                + applicationId + ". New expire time will be " + newTimeout);
        
        Show
        jianhe Jian He added a comment - minor comment on the logging sysout.println( "Updating timeout of an application " + applicationId); sysout.println( "Successfully updated timeout of an application " + applicationId + ". New expire time will be " + newTimeout); may be explicitly mention the timeoutType ? like: sysout.println( "Updating " + timeoutType + " of an application " + applicationId); sysout.println( "Successfully updated " + timeoutType + " of an application " + applicationId + ". New expire time will be " + newTimeout);
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 19s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
        0 mvndep 0m 9s Maven dependency ordering for branch
        +1 mvninstall 7m 17s trunk passed
        +1 compile 5m 14s trunk passed
        +1 checkstyle 0m 50s trunk passed
        +1 mvnsite 2m 26s trunk passed
        +1 mvneclipse 1m 18s trunk passed
        +1 findbugs 4m 5s trunk passed
        +1 javadoc 1m 49s trunk passed
        0 mvndep 0m 11s Maven dependency ordering for patch
        +1 mvninstall 1m 55s the patch passed
        +1 compile 5m 4s the patch passed
        +1 cc 5m 4s the patch passed
        +1 javac 5m 4s the patch passed
        -0 checkstyle 0m 48s hadoop-yarn-project/hadoop-yarn: The patch generated 13 new + 374 unchanged - 2 fixed = 387 total (was 376)
        +1 mvnsite 2m 26s the patch passed
        +1 mvneclipse 1m 17s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 4m 26s the patch passed
        -1 javadoc 0m 23s hadoop-yarn-api in the patch failed.
        +1 unit 0m 33s hadoop-yarn-api in the patch passed.
        +1 unit 2m 26s hadoop-yarn-common in the patch passed.
        +1 unit 43m 2s hadoop-yarn-server-resourcemanager in the patch passed.
        +1 unit 16m 31s hadoop-yarn-client in the patch passed.
        +1 asflicense 0m 31s The patch does not generate ASF License warnings.
        112m 58s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue YARN-4206
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12839780/YARN-4206.2.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux 9bf225f3c3a1 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / c68dad1
        Default Java 1.8.0_111
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13992/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13992/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13992/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13992/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 9s Maven dependency ordering for branch +1 mvninstall 7m 17s trunk passed +1 compile 5m 14s trunk passed +1 checkstyle 0m 50s trunk passed +1 mvnsite 2m 26s trunk passed +1 mvneclipse 1m 18s trunk passed +1 findbugs 4m 5s trunk passed +1 javadoc 1m 49s trunk passed 0 mvndep 0m 11s Maven dependency ordering for patch +1 mvninstall 1m 55s the patch passed +1 compile 5m 4s the patch passed +1 cc 5m 4s the patch passed +1 javac 5m 4s the patch passed -0 checkstyle 0m 48s hadoop-yarn-project/hadoop-yarn: The patch generated 13 new + 374 unchanged - 2 fixed = 387 total (was 376) +1 mvnsite 2m 26s the patch passed +1 mvneclipse 1m 17s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 4m 26s the patch passed -1 javadoc 0m 23s hadoop-yarn-api in the patch failed. +1 unit 0m 33s hadoop-yarn-api in the patch passed. +1 unit 2m 26s hadoop-yarn-common in the patch passed. +1 unit 43m 2s hadoop-yarn-server-resourcemanager in the patch passed. +1 unit 16m 31s hadoop-yarn-client in the patch passed. +1 asflicense 0m 31s The patch does not generate ASF License warnings. 112m 58s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-4206 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12839780/YARN-4206.2.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 9bf225f3c3a1 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / c68dad1 Default Java 1.8.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13992/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13992/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13992/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/13992/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        Updating patch fixing Jian Review comments and few checkstyle and test case failures.

        Currently, updated patch do not consider default value and expect user to input timeout type mandatary. We might feel sub option -updateType is not required as of now since only single timeout considered. But it is better to keep it now considering user facing do not change later if more timeouts supported. May be we can define default value when more timeouts are supported?

        Show
        rohithsharma Rohith Sharma K S added a comment - Updating patch fixing Jian Review comments and few checkstyle and test case failures. Currently, updated patch do not consider default value and expect user to input timeout type mandatary. We might feel sub option -updateType is not required as of now since only single timeout considered. But it is better to keep it now considering user facing do not change later if more timeouts supported. May be we can define default value when more timeouts are supported?
        Hide
        sunilg Sunil G added a comment -

        Rohith Sharma K S
        I think we can take the default behavior to LIFETIME even if there are no other sub-options with updateType.

        Show
        sunilg Sunil G added a comment - Rohith Sharma K S I think we can take the default behavior to LIFETIME even if there are no other sub-options with updateType .
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        updateTimeout, maybe call it updateLifeTimeout?

        How about passing flag, this can be resued in future without changing CLI interface.
        Ex : ./yarn application -updateTimeout 10 -updateType LIFETIME

        Show
        rohithsharma Rohith Sharma K S added a comment - updateTimeout, maybe call it updateLifeTimeout? How about passing flag, this can be resued in future without changing CLI interface. Ex : ./yarn application -updateTimeout 10 -updateType LIFETIME
        Hide
        jianhe Jian He added a comment - - edited

        looks good overall ,thanks Rohith, few comments

        • what is this code for, we can remove?
                if (this.applicationTimeouts.isEmpty()) {
          
                } else {
          
                }
          
        • could you add comments for the API in ApplicationTimeout ?
        • updateTimeout, maybe call it updateLifeTimeout?
        Show
        jianhe Jian He added a comment - - edited looks good overall ,thanks Rohith, few comments what is this code for, we can remove? if ( this .applicationTimeouts.isEmpty()) { } else { } could you add comments for the API in ApplicationTimeout ? updateTimeout, maybe call it updateLifeTimeout?
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 17s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        0 mvndep 0m 11s Maven dependency ordering for branch
        +1 mvninstall 8m 28s trunk passed
        +1 compile 4m 58s trunk passed
        +1 checkstyle 0m 49s trunk passed
        +1 mvnsite 2m 29s trunk passed
        +1 mvneclipse 1m 19s trunk passed
        +1 findbugs 3m 58s trunk passed
        +1 javadoc 1m 48s trunk passed
        0 mvndep 0m 11s Maven dependency ordering for patch
        +1 mvninstall 1m 51s the patch passed
        +1 compile 4m 54s the patch passed
        +1 cc 4m 54s the patch passed
        +1 javac 4m 54s the patch passed
        -0 checkstyle 0m 49s hadoop-yarn-project/hadoop-yarn: The patch generated 16 new + 368 unchanged - 1 fixed = 384 total (was 369)
        +1 mvnsite 2m 32s the patch passed
        +1 mvneclipse 1m 27s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 5m 13s the patch passed
        +1 javadoc 2m 1s the patch passed
        +1 unit 0m 40s hadoop-yarn-api in the patch passed.
        -1 unit 3m 0s hadoop-yarn-common in the patch failed.
        -1 unit 49m 34s hadoop-yarn-server-resourcemanager in the patch failed.
        -1 unit 17m 17s hadoop-yarn-client in the patch failed.
        +1 asflicense 0m 32s The patch does not generate ASF License warnings.
        123m 16s



        Reason Tests
        Failed junit tests hadoop.yarn.api.TestPBImplRecords
          hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer
          hadoop.yarn.server.resourcemanager.rmapp.TestApplicationLifetimeMonitor
          hadoop.yarn.client.cli.TestYarnCLI



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue YARN-4206
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12839398/YARN-4506.1.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux efd4d6b2bfc0 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / b2d4b7b
        Default Java 1.8.0_111
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13957/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/13957/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/13957/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/13957/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13957/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13957/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 11s Maven dependency ordering for branch +1 mvninstall 8m 28s trunk passed +1 compile 4m 58s trunk passed +1 checkstyle 0m 49s trunk passed +1 mvnsite 2m 29s trunk passed +1 mvneclipse 1m 19s trunk passed +1 findbugs 3m 58s trunk passed +1 javadoc 1m 48s trunk passed 0 mvndep 0m 11s Maven dependency ordering for patch +1 mvninstall 1m 51s the patch passed +1 compile 4m 54s the patch passed +1 cc 4m 54s the patch passed +1 javac 4m 54s the patch passed -0 checkstyle 0m 49s hadoop-yarn-project/hadoop-yarn: The patch generated 16 new + 368 unchanged - 1 fixed = 384 total (was 369) +1 mvnsite 2m 32s the patch passed +1 mvneclipse 1m 27s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 5m 13s the patch passed +1 javadoc 2m 1s the patch passed +1 unit 0m 40s hadoop-yarn-api in the patch passed. -1 unit 3m 0s hadoop-yarn-common in the patch failed. -1 unit 49m 34s hadoop-yarn-server-resourcemanager in the patch failed. -1 unit 17m 17s hadoop-yarn-client in the patch failed. +1 asflicense 0m 32s The patch does not generate ASF License warnings. 123m 16s Reason Tests Failed junit tests hadoop.yarn.api.TestPBImplRecords   hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer   hadoop.yarn.server.resourcemanager.rmapp.TestApplicationLifetimeMonitor   hadoop.yarn.client.cli.TestYarnCLI Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-4206 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12839398/YARN-4506.1.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux efd4d6b2bfc0 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / b2d4b7b Default Java 1.8.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13957/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/13957/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/13957/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/13957/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13957/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/13957/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        Updating patch for supporting YarnClient API, ApplicationCLI to print timeout in status command , support for new option -updateTimeout. And modification is in ApplicationReport to get list of timeouts values.

        Show
        rohithsharma Rohith Sharma K S added a comment - Updating patch for supporting YarnClient API, ApplicationCLI to print timeout in status command , support for new option -updateTimeout. And modification is in ApplicationReport to get list of timeouts values.
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        In this JIRA, lets handle ApplicationReport changes and CLI part. For the yarn web UI, it is required to change for both new and old UI, so lets handle it YARN-5650 separately. I will update JIRA summary to reflect the same.

        Show
        rohithsharma Rohith Sharma K S added a comment - In this JIRA, lets handle ApplicationReport changes and CLI part. For the yarn web UI, it is required to change for both new and old UI, so lets handle it YARN-5650 separately. I will update JIRA summary to reflect the same.
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        ahh.. In LIFETIME monitoring, application attempt might get pre-empted, but lifetime timeout is considered from app submitted to app finished. So, in between any number of application-attempt can launch and preempt, its not an issue because app state is still running. Above mentioned(your comment) scenario need to think of for other types of timeout that we are planning to support.

        Show
        rohithsharma Rohith Sharma K S added a comment - ahh.. In LIFETIME monitoring, application attempt might get pre-empted, but lifetime timeout is considered from app submitted to app finished. So, in between any number of application-attempt can launch and preempt, its not an issue because app state is still running. Above mentioned(your comment) scenario need to think of for other types of timeout that we are planning to support.
        Hide
        gsaha Gour Saha added a comment -

        When the app is back into RUNNING state at 10:10:35 what will the value of remaining timeout and absolute timeout be?

        Show
        gsaha Gour Saha added a comment - When the app is back into RUNNING state at 10:10:35 what will the value of remaining timeout and absolute timeout be?
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        Thats a good question For the applications which are completed or applications which are not enabled with timeout, report should have special value i.e -1 which indicates either application is completed or timeout not enabled. In your case, timeout value will be set to -1 which indicates application is in completing states.

        Show
        rohithsharma Rohith Sharma K S added a comment - Thats a good question For the applications which are completed or applications which are not enabled with timeout, report should have special value i.e -1 which indicates either application is completed or timeout not enabled. In your case, timeout value will be set to -1 which indicates application is in completing states.
        Hide
        gsaha Gour Saha added a comment -

        What happens when the app gets pre-empted at 10:10:15 for 20 secs?

        Show
        gsaha Gour Saha added a comment - What happens when the app gets pre-empted at 10:10:15 for 20 secs?
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        Isn't absolute timeout going to be inaccurate?

        No, It will be fixed time. Lets say, I have submitted application with 10seconds timeout. Assume job submitted time is 10:10:10 PM. So absolute time for application to get timed out is 10:10:20. After 3 seconds, if user request for app-report, then both 10:10:20 PM as absolute time and 7 secs as remaining time could be sent. Absolute time never changes, but remaining time changes on each user request.

        Show
        rohithsharma Rohith Sharma K S added a comment - Isn't absolute timeout going to be inaccurate? No, It will be fixed time. Lets say, I have submitted application with 10seconds timeout. Assume job submitted time is 10:10:10 PM. So absolute time for application to get timed out is 10:10:20. After 3 seconds, if user request for app-report, then both 10:10:20 PM as absolute time and 7 secs as remaining time could be sent. Absolute time never changes, but remaining time changes on each user request.
        Hide
        gsaha Gour Saha added a comment -

        Isn't absolute timeout going to be inaccurate? Given that we know the remaining timeout of an application, what we don't know is that going forward will it be pre-empted or not. If it is, then the absolute value will move further down? Is my understanding incorrect?

        Show
        gsaha Gour Saha added a comment - Isn't absolute timeout going to be inaccurate? Given that we know the remaining timeout of an application, what we don't know is that going forward will it be pre-empted or not. If it is, then the absolute value will move further down? Is my understanding incorrect?
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        I don't think we need an additional API in ApplicationClientProtocol to get that.

        Thats true. We do not really require new client API.

        Additionally If I look at the user use cases, user would require both absolute time and remaining time. Say if timeout is less than a day or half a day, then remaining timeout would be sufficient. If timeout is 3-4 days later, user might interested in knowing which day it will get time out. So, How about new object ApplicationTimeout which gives both absolute-time and remaining-time for given ApplicationTimeoutType? The ApplicationReport would consists of Map<ApplicationTimeoutTypes, ApplicationTimeout>. Thoughts?

        Show
        rohithsharma Rohith Sharma K S added a comment - I don't think we need an additional API in ApplicationClientProtocol to get that. Thats true. We do not really require new client API. Additionally If I look at the user use cases, user would require both absolute time and remaining time. Say if timeout is less than a day or half a day, then remaining timeout would be sufficient. If timeout is 3-4 days later, user might interested in knowing which day it will get time out. So, How about new object ApplicationTimeout which gives both absolute-time and remaining-time for given ApplicationTimeoutType? The ApplicationReport would consists of Map<ApplicationTimeoutTypes, ApplicationTimeout>. Thoughts?
        Hide
        jianhe Jian He added a comment -

        The remaining time can be inferred from the absolute timeout value ? If so, I don't think we need an additional API in ApplicationClientProtocol to get that.

        Show
        jianhe Jian He added a comment - The remaining time can be inferred from the absolute timeout value ? If so, I don't think we need an additional API in ApplicationClientProtocol to get that.
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        Thanks Gour Saha for bringing out point remaining time for an application. I think application-report should contains absoluteTimeout value i.e when it gets expired. To the client, there could be new API ApplicationClientProtocol#getApplicationRemainingTime(ApplicationId) which gets remaining timeout in seconds. Thoughts? Sunil G Jian He Vinod Kumar Vavilapalli

        Show
        rohithsharma Rohith Sharma K S added a comment - Thanks Gour Saha for bringing out point remaining time for an application. I think application-report should contains absoluteTimeout value i.e when it gets expired. To the client, there could be new API ApplicationClientProtocol#getApplicationRemainingTime(ApplicationId) which gets remaining timeout in seconds. Thoughts? Sunil G Jian He Vinod Kumar Vavilapalli
        Hide
        gsaha Gour Saha added a comment -

        It will be very helpful from an end-user perspective to provide the remaining-timeouts of the application in the report. For example, remaining-lifetime of an application will let the app-owner understand that he/she needs to extend the lifetime because the initial estimate of completing a task was not accurate. Note, timeouts update is going to supported as per YARN-5611.

        It might not be easy or straightforward to calculate the remaining-timeouts for all ApplicationTimeoutTypes (we now support multiple timeouts). However we should try to provide them for whichever ones we can. The reason I am adding this note is because, we should not discard the idea of providing the remaining-timeouts, just because remaining-timeout for one of the ApplicationTimeoutType cannot be calculated or does not makes sense.

        Show
        gsaha Gour Saha added a comment - It will be very helpful from an end-user perspective to provide the remaining-timeouts of the application in the report. For example, remaining-lifetime of an application will let the app-owner understand that he/she needs to extend the lifetime because the initial estimate of completing a task was not accurate. Note, timeouts update is going to supported as per YARN-5611 . It might not be easy or straightforward to calculate the remaining-timeouts for all ApplicationTimeoutTypes (we now support multiple timeouts). However we should try to provide them for whichever ones we can. The reason I am adding this note is because, we should not discard the idea of providing the remaining-timeouts, just because remaining-timeout for one of the ApplicationTimeoutType cannot be calculated or does not makes sense.

          People

          • Assignee:
            rohithsharma Rohith Sharma K S
            Reporter:
            nijel nijel
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development