Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha2
    • Component/s: scheduler
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      AppTimeoutInfo is exposed to REST clients. This object request parameters should be validated before processing it to clientRMService.

      And also handles couple of minor issues in REST services.

      1. YARN-5970-branch-2.0001.patch
        7 kB
        Rohith Sharma K S
      2. YARN-5970.0.patch
        7 kB
        Rohith Sharma K S

        Activity

        Hide
        sunilg Sunil G added a comment -

        Thanks Rohith Sharma K S for the contribution and thanks Jian He for additional reviews. Committed to trunk and branch-2.

        Show
        sunilg Sunil G added a comment - Thanks Rohith Sharma K S for the contribution and thanks Jian He for additional reviews. Committed to trunk and branch-2.
        Hide
        sunilg Sunil G added a comment -

        Test case failures looks unrelated and already tracked via YARN-5652, YARN-5548, YARN-5816.
        branch-2 patch looks good. Committing the same.

        Show
        sunilg Sunil G added a comment - Test case failures looks unrelated and already tracked via YARN-5652 , YARN-5548 , YARN-5816 . branch-2 patch looks good. Committing the same.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 19s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 29s Maven dependency ordering for branch
        +1 mvninstall 6m 44s branch-2 passed
        +1 compile 2m 0s branch-2 passed with JDK v1.8.0_111
        +1 compile 2m 19s branch-2 passed with JDK v1.7.0_121
        +1 checkstyle 0m 41s branch-2 passed
        +1 mvnsite 1m 13s branch-2 passed
        +1 mvneclipse 0m 31s branch-2 passed
        +1 findbugs 2m 38s branch-2 passed
        +1 javadoc 1m 1s branch-2 passed with JDK v1.8.0_111
        +1 javadoc 1m 9s branch-2 passed with JDK v1.7.0_121
        0 mvndep 0m 13s Maven dependency ordering for patch
        +1 mvninstall 1m 12s the patch passed
        +1 compile 2m 27s the patch passed with JDK v1.8.0_111
        +1 javac 2m 27s the patch passed
        +1 compile 2m 33s the patch passed with JDK v1.7.0_121
        +1 javac 2m 33s the patch passed
        +1 checkstyle 0m 46s the patch passed
        +1 mvnsite 1m 19s the patch passed
        +1 mvneclipse 0m 34s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 3m 22s the patch passed
        +1 javadoc 0m 56s the patch passed with JDK v1.8.0_111
        +1 javadoc 1m 8s the patch passed with JDK v1.7.0_121
        +1 unit 2m 49s hadoop-yarn-common in the patch passed with JDK v1.7.0_121.
        -1 unit 45m 19s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_121.
        +1 asflicense 0m 21s The patch does not generate ASF License warnings.
        140m 46s



        Reason Tests
        JDK v1.7.0_121 Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart
          hadoop.yarn.server.resourcemanager.TestRMAdminService
          hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:b59b8b7
        JIRA Issue YARN-5970
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12842556/YARN-5970-branch-2.0001.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux eefae5386f10 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision branch-2 / 73d5de5
        Default Java 1.7.0_121
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_111 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_121
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-YARN-Build/14241/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_121.txt
        JDK v1.7.0_121 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14241/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/14241/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 29s Maven dependency ordering for branch +1 mvninstall 6m 44s branch-2 passed +1 compile 2m 0s branch-2 passed with JDK v1.8.0_111 +1 compile 2m 19s branch-2 passed with JDK v1.7.0_121 +1 checkstyle 0m 41s branch-2 passed +1 mvnsite 1m 13s branch-2 passed +1 mvneclipse 0m 31s branch-2 passed +1 findbugs 2m 38s branch-2 passed +1 javadoc 1m 1s branch-2 passed with JDK v1.8.0_111 +1 javadoc 1m 9s branch-2 passed with JDK v1.7.0_121 0 mvndep 0m 13s Maven dependency ordering for patch +1 mvninstall 1m 12s the patch passed +1 compile 2m 27s the patch passed with JDK v1.8.0_111 +1 javac 2m 27s the patch passed +1 compile 2m 33s the patch passed with JDK v1.7.0_121 +1 javac 2m 33s the patch passed +1 checkstyle 0m 46s the patch passed +1 mvnsite 1m 19s the patch passed +1 mvneclipse 0m 34s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 22s the patch passed +1 javadoc 0m 56s the patch passed with JDK v1.8.0_111 +1 javadoc 1m 8s the patch passed with JDK v1.7.0_121 +1 unit 2m 49s hadoop-yarn-common in the patch passed with JDK v1.7.0_121. -1 unit 45m 19s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_121. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 140m 46s Reason Tests JDK v1.7.0_121 Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart   hadoop.yarn.server.resourcemanager.TestRMAdminService   hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Issue YARN-5970 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12842556/YARN-5970-branch-2.0001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux eefae5386f10 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / 73d5de5 Default Java 1.7.0_121 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_111 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-YARN-Build/14241/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_121.txt JDK v1.7.0_121 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14241/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/14241/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        uploaded branch-2 patch

        Show
        rohithsharma Rohith Sharma K S added a comment - uploaded branch-2 patch
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10968 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10968/)
        YARN-5970. Validate application update timeout request parameters. (sunilg: rev 74d0066d3392169bec872f438a0818e2f5323010)

        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/Times.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMServerUtils.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesAppsModification.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10968 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10968/ ) YARN-5970 . Validate application update timeout request parameters. (sunilg: rev 74d0066d3392169bec872f438a0818e2f5323010) (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/Times.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMServerUtils.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesAppsModification.java
        Hide
        sunilg Sunil G added a comment -

        Rohith Sharma K S
        Thank for the patch. I committed to trunk. It seems there is a conflict for branch-2. please help to share a branch-2 patch.

        Show
        sunilg Sunil G added a comment - Rohith Sharma K S Thank for the patch. I committed to trunk. It seems there is a conflict for branch-2. please help to share a branch-2 patch.
        Hide
        sunilg Sunil G added a comment -

        Patch looks good for me. I will commit the patch later today if there are no objections.

        Show
        sunilg Sunil G added a comment - Patch looks good for me. I will commit the patch later today if there are no objections.
        Hide
        sunilg Sunil G added a comment -

        This value can not be seconds. This value be in-valid ISO8601 format. so, we need to print invalid configured value that can be anything.

        Make sense. I earlier got confused by the extended message in the patch.

        Show
        sunilg Sunil G added a comment - This value can not be seconds. This value be in-valid ISO8601 format. so, we need to print invalid configured value that can be anything. Make sense. I earlier got confused by the extended message in the patch.
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        Since we do not know the error offset, do we need to set it to -1?

        offset is mandatory for parseexcpetion. However, since null value is being passed, offset remain as -1.

        Could we append "seconds" as well at the end.

        This value can not be seconds. This value be in-valid ISO8601 format. so, we need to print invalid configured value that can be anything.

        think point 3 is handled in YARN-5965

        yes, yarn-5965 handles comment-3.

        Show
        rohithsharma Rohith Sharma K S added a comment - Since we do not know the error offset, do we need to set it to -1? offset is mandatory for parseexcpetion. However, since null value is being passed, offset remain as -1. Could we append "seconds" as well at the end. This value can not be seconds. This value be in-valid ISO8601 format. so, we need to print invalid configured value that can be anything. think point 3 is handled in YARN-5965 yes, yarn-5965 handles comment-3.
        Hide
        sunilg Sunil G added a comment -

        I think point 3 is handled in YARN-5965. Pls skip my 3rd comment.

        Show
        sunilg Sunil G added a comment - I think point 3 is handled in YARN-5965 . Pls skip my 3rd comment.
        Hide
        sunilg Sunil G added a comment -

        Thanks Rohith Sharma K S

        1)

        throw new ParseException("Invalid input.", -1);

        Since we do not know the error offset, do we need to set it to -1?

        2)

        "Expire time is not in ISO8601 format. ISO8601 supported "
         + "format is yyyy-MM-dd'T'HH:mm:ss.SSSZ. Configured "
         + "timeout value is " + timeout.getValue();
        

        Could we append "seconds" as well at the end.

        3) When app is finished before timeout, could we validate ? I think we can club that validation also here. Thoughts?

        Show
        sunilg Sunil G added a comment - Thanks Rohith Sharma K S 1) throw new ParseException("Invalid input.", -1); Since we do not know the error offset, do we need to set it to -1? 2) "Expire time is not in ISO8601 format. ISO8601 supported " + "format is yyyy-MM-dd'T'HH:mm:ss.SSSZ. Configured " + "timeout value is " + timeout.getValue(); Could we append "seconds" as well at the end. 3) When app is finished before timeout, could we validate ? I think we can club that validation also here. Thoughts?
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 17s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 10s Maven dependency ordering for branch
        +1 mvninstall 7m 5s trunk passed
        +1 compile 5m 13s trunk passed
        +1 checkstyle 0m 46s trunk passed
        +1 mvnsite 1m 22s trunk passed
        +1 mvneclipse 0m 43s trunk passed
        +1 findbugs 2m 10s trunk passed
        +1 javadoc 1m 2s trunk passed
        0 mvndep 0m 10s Maven dependency ordering for patch
        +1 mvninstall 0m 59s the patch passed
        +1 compile 4m 41s the patch passed
        +1 javac 4m 41s the patch passed
        +1 checkstyle 0m 46s the patch passed
        +1 mvnsite 1m 21s the patch passed
        +1 mvneclipse 0m 43s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 2m 33s the patch passed
        +1 javadoc 1m 1s the patch passed
        +1 unit 2m 26s hadoop-yarn-common in the patch passed.
        -1 unit 43m 57s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 asflicense 0m 28s The patch does not generate ASF License warnings.
        86m 1s



        Reason Tests
        Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue YARN-5970
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12841938/YARN-5970.0.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 12a53cb9c630 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 7b988e8
        Default Java 1.8.0_111
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-YARN-Build/14195/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14195/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/14195/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 10s Maven dependency ordering for branch +1 mvninstall 7m 5s trunk passed +1 compile 5m 13s trunk passed +1 checkstyle 0m 46s trunk passed +1 mvnsite 1m 22s trunk passed +1 mvneclipse 0m 43s trunk passed +1 findbugs 2m 10s trunk passed +1 javadoc 1m 2s trunk passed 0 mvndep 0m 10s Maven dependency ordering for patch +1 mvninstall 0m 59s the patch passed +1 compile 4m 41s the patch passed +1 javac 4m 41s the patch passed +1 checkstyle 0m 46s the patch passed +1 mvnsite 1m 21s the patch passed +1 mvneclipse 0m 43s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 33s the patch passed +1 javadoc 1m 1s the patch passed +1 unit 2m 26s hadoop-yarn-common in the patch passed. -1 unit 43m 57s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 28s The patch does not generate ASF License warnings. 86m 1s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-5970 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12841938/YARN-5970.0.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 12a53cb9c630 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 7b988e8 Default Java 1.8.0_111 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-YARN-Build/14195/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14195/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/14195/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        Updating the patch with following changes.

        1. Validate for expiryTime with null value.
        2. As per comment from Gour, changes the REST URI to /apps/$appid/timeouts/$type for GET request.
        Show
        rohithsharma Rohith Sharma K S added a comment - Updating the patch with following changes. Validate for expiryTime with null value. As per comment from Gour, changes the REST URI to /apps/$appid/ timeouts /$type for GET request.

          People

          • Assignee:
            rohithsharma Rohith Sharma K S
            Reporter:
            rohithsharma Rohith Sharma K S
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development