Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3410

YARN admin should be able to remove individual application records from RMStateStore

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: resourcemanager, yarn
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      When RM state store entered an unexpected state, one example is YARN-2340, when an attempt is not in final state but app already completed, RM can never get up unless format RMStateStore.

      I think we should support remove individual application records from RMStateStore to unblock RM admin make choice of either waiting for a fix or format state store.

      In addition, RM should be able to report all fatal errors (which will shutdown RM) when doing app recovery, this can save admin some time to remove apps in bad state.

      1. 0001-YARN-3410.patch
        16 kB
        Rohith Sharma K S
      2. 0001-YARN-3410.patch
        16 kB
        Rohith Sharma K S
      3. 0001-YARN-3410-v1.patch
        16 kB
        Rohith Sharma K S
      4. 0002-YARN-3410.patch
        16 kB
        Rohith Sharma K S
      5. 0003-YARN-3410.patch
        17 kB
        Rohith Sharma K S
      6. 0004-YARN-3410.patch
        17 kB
        Rohith Sharma K S
      7. 0004-YARN-3410-addendum.patch
        1 kB
        Rohith Sharma K S
      8. 0004-YARN-3410-addendum-branch-2.patch
        1 kB
        Rohith Sharma K S
      9. 0004-YARN-3410-branch-2.patch
        17 kB
        Rohith Sharma K S

        Issue Links

          Activity

          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2121 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2121/)
          YARN-3410. YARN admin should be able to remove individual application records from RMStateStore. (Rohith Sharmaks via wangda) (wangda: rev e71d0d87d9b388f211a8eb3d2cd9af347abf9bda)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestLeveldbRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/FileSystemRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/LeveldbRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/bin/yarn
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/NullRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/MemoryRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestFSRMStateStore.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStoreTestBase.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/YarnCommands.md
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStore.java
            YARN-3410. Addendum fix for compilation error. Contributed by Rohith. (aajisaka: rev b08908ae5eaf60a7fc70bf60493a533e915553c5)
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/YarnCommands.md
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2121 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2121/ ) YARN-3410 . YARN admin should be able to remove individual application records from RMStateStore. (Rohith Sharmaks via wangda) (wangda: rev e71d0d87d9b388f211a8eb3d2cd9af347abf9bda) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestLeveldbRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/FileSystemRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/LeveldbRMStateStore.java hadoop-yarn-project/hadoop-yarn/bin/yarn hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/NullRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/MemoryRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestFSRMStateStore.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStoreTestBase.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/YarnCommands.md hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStore.java YARN-3410 . Addendum fix for compilation error. Contributed by Rohith. (aajisaka: rev b08908ae5eaf60a7fc70bf60493a533e915553c5) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/YarnCommands.md
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #172 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/172/)
          YARN-3410. YARN admin should be able to remove individual application records from RMStateStore. (Rohith Sharmaks via wangda) (wangda: rev e71d0d87d9b388f211a8eb3d2cd9af347abf9bda)

          • hadoop-yarn-project/hadoop-yarn/bin/yarn
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/LeveldbRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestFSRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestLeveldbRMStateStore.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/MemoryRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/YarnCommands.md
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/FileSystemRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStoreTestBase.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/NullRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStore.java
            YARN-3410. Addendum fix for compilation error. Contributed by Rohith. (aajisaka: rev b08908ae5eaf60a7fc70bf60493a533e915553c5)
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/YarnCommands.md
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #172 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/172/ ) YARN-3410 . YARN admin should be able to remove individual application records from RMStateStore. (Rohith Sharmaks via wangda) (wangda: rev e71d0d87d9b388f211a8eb3d2cd9af347abf9bda) hadoop-yarn-project/hadoop-yarn/bin/yarn hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/LeveldbRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestFSRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestLeveldbRMStateStore.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/MemoryRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/YarnCommands.md hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/FileSystemRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStoreTestBase.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/NullRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStore.java YARN-3410 . Addendum fix for compilation error. Contributed by Rohith. (aajisaka: rev b08908ae5eaf60a7fc70bf60493a533e915553c5) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/YarnCommands.md
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk #905 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/905/)
          YARN-3410. YARN admin should be able to remove individual application records from RMStateStore. (Rohith Sharmaks via wangda) (wangda: rev e71d0d87d9b388f211a8eb3d2cd9af347abf9bda)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStoreTestBase.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/FileSystemRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/LeveldbRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/NullRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/MemoryRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestFSRMStateStore.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/YarnCommands.md
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestLeveldbRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/bin/yarn
            YARN-3410. Addendum fix for compilation error. Contributed by Rohith. (aajisaka: rev b08908ae5eaf60a7fc70bf60493a533e915553c5)
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/YarnCommands.md
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #905 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/905/ ) YARN-3410 . YARN admin should be able to remove individual application records from RMStateStore. (Rohith Sharmaks via wangda) (wangda: rev e71d0d87d9b388f211a8eb3d2cd9af347abf9bda) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStoreTestBase.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/FileSystemRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/LeveldbRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/NullRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/MemoryRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestFSRMStateStore.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/YarnCommands.md hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestLeveldbRMStateStore.java hadoop-yarn-project/hadoop-yarn/bin/yarn YARN-3410 . Addendum fix for compilation error. Contributed by Rohith. (aajisaka: rev b08908ae5eaf60a7fc70bf60493a533e915553c5) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/YarnCommands.md
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #171 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/171/)
          YARN-3410. YARN admin should be able to remove individual application records from RMStateStore. (Rohith Sharmaks via wangda) (wangda: rev e71d0d87d9b388f211a8eb3d2cd9af347abf9bda)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/FileSystemRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/YarnCommands.md
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/NullRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/MemoryRMStateStore.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestFSRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/LeveldbRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestLeveldbRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStoreTestBase.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/bin/yarn
            YARN-3410. Addendum fix for compilation error. Contributed by Rohith. (aajisaka: rev b08908ae5eaf60a7fc70bf60493a533e915553c5)
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/YarnCommands.md
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #171 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/171/ ) YARN-3410 . YARN admin should be able to remove individual application records from RMStateStore. (Rohith Sharmaks via wangda) (wangda: rev e71d0d87d9b388f211a8eb3d2cd9af347abf9bda) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/FileSystemRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/YarnCommands.md hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/NullRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/MemoryRMStateStore.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestFSRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/LeveldbRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestLeveldbRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStoreTestBase.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java hadoop-yarn-project/hadoop-yarn/bin/yarn YARN-3410 . Addendum fix for compilation error. Contributed by Rohith. (aajisaka: rev b08908ae5eaf60a7fc70bf60493a533e915553c5) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/YarnCommands.md
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #162 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/162/)
          YARN-3410. YARN admin should be able to remove individual application records from RMStateStore. (Rohith Sharmaks via wangda) (wangda: rev e71d0d87d9b388f211a8eb3d2cd9af347abf9bda)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/MemoryRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/bin/yarn
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/YarnCommands.md
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/FileSystemRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/LeveldbRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestFSRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStoreTestBase.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestLeveldbRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/NullRMStateStore.java
            YARN-3410. Addendum fix for compilation error. Contributed by Rohith. (aajisaka: rev b08908ae5eaf60a7fc70bf60493a533e915553c5)
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/YarnCommands.md
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #162 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/162/ ) YARN-3410 . YARN admin should be able to remove individual application records from RMStateStore. (Rohith Sharmaks via wangda) (wangda: rev e71d0d87d9b388f211a8eb3d2cd9af347abf9bda) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/MemoryRMStateStore.java hadoop-yarn-project/hadoop-yarn/bin/yarn hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/YarnCommands.md hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/FileSystemRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/LeveldbRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestFSRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStoreTestBase.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestLeveldbRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/NullRMStateStore.java YARN-3410 . Addendum fix for compilation error. Contributed by Rohith. (aajisaka: rev b08908ae5eaf60a7fc70bf60493a533e915553c5) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/YarnCommands.md
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2103 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2103/)
          YARN-3410. YARN admin should be able to remove individual application records from RMStateStore. (Rohith Sharmaks via wangda) (wangda: rev e71d0d87d9b388f211a8eb3d2cd9af347abf9bda)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/FileSystemRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestLeveldbRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestFSRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/bin/yarn
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/MemoryRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/YarnCommands.md
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/LeveldbRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/NullRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStoreTestBase.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
            YARN-3410. Addendum fix for compilation error. Contributed by Rohith. (aajisaka: rev b08908ae5eaf60a7fc70bf60493a533e915553c5)
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/YarnCommands.md
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2103 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2103/ ) YARN-3410 . YARN admin should be able to remove individual application records from RMStateStore. (Rohith Sharmaks via wangda) (wangda: rev e71d0d87d9b388f211a8eb3d2cd9af347abf9bda) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/FileSystemRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestLeveldbRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestFSRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStore.java hadoop-yarn-project/hadoop-yarn/bin/yarn hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/MemoryRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/YarnCommands.md hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/LeveldbRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/NullRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStoreTestBase.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java YARN-3410 . Addendum fix for compilation error. Contributed by Rohith. (aajisaka: rev b08908ae5eaf60a7fc70bf60493a533e915553c5) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/YarnCommands.md
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          I am surprised that how jenkins misses documentation compilation issue when there is change in the documentation package. It would be good if jenkins able to identify such issue while running HadoopQA.

          Show
          rohithsharma Rohith Sharma K S added a comment - I am surprised that how jenkins misses documentation compilation issue when there is change in the documentation package. It would be good if jenkins able to identify such issue while running HadoopQA.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #7635 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7635/)
          YARN-3410. Addendum fix for compilation error. Contributed by Rohith. (aajisaka: rev b08908ae5eaf60a7fc70bf60493a533e915553c5)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/YarnCommands.md
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7635 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7635/ ) YARN-3410 . Addendum fix for compilation error. Contributed by Rohith. (aajisaka: rev b08908ae5eaf60a7fc70bf60493a533e915553c5) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/YarnCommands.md
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Committed the addendum patch to trunk and branch-2. Thanks Rohith Sharma K S and Brahma Reddy Battula for contribution.

          Show
          ajisakaa Akira Ajisaka added a comment - Committed the addendum patch to trunk and branch-2. Thanks Rohith Sharma K S and Brahma Reddy Battula for contribution.
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          Thanks Akira Ajisaka for quick review

          Show
          rohithsharma Rohith Sharma K S added a comment - Thanks Akira Ajisaka for quick review
          Hide
          ajisakaa Akira Ajisaka added a comment -

          +1, the addendum patch looks good to me.

          I applied the patch and executed 'mvn site' to compile the document.

          Show
          ajisakaa Akira Ajisaka added a comment - +1, the addendum patch looks good to me. I applied the patch and executed 'mvn site' to compile the document.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          +1, the addendum patch looks good to me. Committing this.

          Show
          ajisakaa Akira Ajisaka added a comment - +1, the addendum patch looks good to me. Committing this.
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          Attached the addendum patch for fixing documentation compilation issue.

          Show
          rohithsharma Rohith Sharma K S added a comment - Attached the addendum patch for fixing documentation compilation issue.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          I will re open this and give add on patch

          Thank you.

          Hi Brahma Reddy Battula, would you file a separate jira to fix the document?

          Now Rohith is going to give add-on patch, so you don't need to open a separate jira.

          Show
          ajisakaa Akira Ajisaka added a comment - I will re open this and give add on patch Thank you. Hi Brahma Reddy Battula, would you file a separate jira to fix the document? Now Rohith is going to give add-on patch, so you don't need to open a separate jira.
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          Thanks Brahma Reddy Battula for pointing out this. I am running locally using 'mvn clean install -DskipTests -Pdoc', it is able to compile.

          I will re open this and give add on patch

          Show
          rohithsharma Rohith Sharma K S added a comment - Thanks Brahma Reddy Battula for pointing out this. I am running locally using 'mvn clean install -DskipTests -Pdoc', it is able to compile. I will re open this and give add on patch
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Hi Brahma Reddy Battula, would you file a separate jira to fix the document?

          Show
          ajisakaa Akira Ajisaka added a comment - Hi Brahma Reddy Battula , would you file a separate jira to fix the document?
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          There is site comipliation error, after this in..

          [ERROR] Failed to execute goal org.apache.maven.plugins:maven-site-plugin:3.4:site (default-site) on project hadoop-yarn-site: Error parsing '/home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/YarnCommands.md': line [251] Error parsing the model: end tag name </td> must match start tag name <appId> from line 251 (position: TEXT seen ...td align="left">-remove-application-from-state-store <appId> </td>... @251:74) -> [Help 1]

          Please check the following link for more..

          https://builds.apache.org/job/PreCommit-HADOOP-Build/6144/artifact/patchprocess/patchSiteWarnings.txt

          Show
          brahmareddy Brahma Reddy Battula added a comment - There is site comipliation error, after this in.. [ERROR] Failed to execute goal org.apache.maven.plugins:maven-site-plugin:3.4:site (default-site) on project hadoop-yarn-site: Error parsing '/home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/YarnCommands.md': line [251] Error parsing the model: end tag name </td> must match start tag name <appId> from line 251 (position: TEXT seen ...td align="left">-remove-application-from-state-store <appId> </td>... @251:74) -> [Help 1] Please check the following link for more.. https://builds.apache.org/job/PreCommit-HADOOP-Build/6144/artifact/patchprocess/patchSiteWarnings.txt
          Show
          brahmareddy Brahma Reddy Battula added a comment - https://builds.apache.org/job/PreCommit-HADOOP-Build/6144/artifact/patchprocess/patchSiteWarnings.txt
          Show
          brahmareddy Brahma Reddy Battula added a comment - https://builds.apache.org/job/PreCommit-HADOOP-Build/6144/artifact/patchprocess/patchSiteWarnings.txt
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #7631 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7631/)
          YARN-3410. YARN admin should be able to remove individual application records from RMStateStore. (Rohith Sharmaks via wangda) (wangda: rev e71d0d87d9b388f211a8eb3d2cd9af347abf9bda)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/NullRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestLeveldbRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStoreTestBase.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/YarnCommands.md
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestFSRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/MemoryRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/FileSystemRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/bin/yarn
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/LeveldbRMStateStore.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #7631 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7631/ ) YARN-3410 . YARN admin should be able to remove individual application records from RMStateStore. (Rohith Sharmaks via wangda) (wangda: rev e71d0d87d9b388f211a8eb3d2cd9af347abf9bda) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/NullRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestLeveldbRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStoreTestBase.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/YarnCommands.md hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestFSRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/MemoryRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/FileSystemRMStateStore.java hadoop-yarn-project/hadoop-yarn/bin/yarn hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/LeveldbRMStateStore.java
          Hide
          leftnoteasy Wangda Tan added a comment -

          Just committed to trunk/branch-2, thanks Rohith Sharma K S and review from Vinod Kumar Vavilapalli/Brahma Reddy Battula! Remove app from state while RM is running issue should be addressed together in YARN-2268.

          Show
          leftnoteasy Wangda Tan added a comment - Just committed to trunk/branch-2, thanks Rohith Sharma K S and review from Vinod Kumar Vavilapalli / Brahma Reddy Battula ! Remove app from state while RM is running issue should be addressed together in YARN-2268 .
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Seems like this is going in first. If not, this should also take care of YARN-2268.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Seems like this is going in first. If not, this should also take care of YARN-2268 .
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12726783/0004-YARN-3410-branch-2.patch
          against trunk revision d52de61.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7416//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12726783/0004-YARN-3410-branch-2.patch against trunk revision d52de61. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7416//console This message is automatically generated.
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          Updated the patch for branch-2

          Show
          rohithsharma Rohith Sharma K S added a comment - Updated the patch for branch-2
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          Its already handled in that patch.

          Show
          rohithsharma Rohith Sharma K S added a comment - Its already handled in that patch.
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          Sure, I will create the patch for branch-2 and upload it.

          Show
          rohithsharma Rohith Sharma K S added a comment - Sure, I will create the patch for branch-2 and upload it.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          004-Patch overall LGTM,, One minor comment..Rohith Sharma K S can you please update the following usage also as Wangda Tan also pointed earlier...

          +  private static void printUsage(PrintStream out) {
          +    out.println("Usage: java ResourceManager [-format-state-store]");
          +    out.println("                            "
          +        + "[-remove-application-from-state-store <appId>]" + "\n");
          +  }
          

          TO

          Usage: yarn resourcemanager [-format-state-store]
                                      [-remove..] <appId>
          
          Show
          brahmareddy Brahma Reddy Battula added a comment - 004-Patch overall LGTM,, One minor comment.. Rohith Sharma K S can you please update the following usage also as Wangda Tan also pointed earlier... + private static void printUsage(PrintStream out) { + out.println( "Usage: java ResourceManager [-format-state-store]" ); + out.println( " " + + "[-remove-application-from-state-store <appId>]" + "\n" ); + } TO Usage: yarn resourcemanager [-format-state-store] [-remove..] <appId>
          Hide
          leftnoteasy Wangda Tan added a comment -

          While committing, I found conflicts happens while applying to branch-2. Rohith Sharma K S, could you upload a branch-2 patch? It's better to commit both of them together.

          Show
          leftnoteasy Wangda Tan added a comment - While committing, I found conflicts happens while applying to branch-2. Rohith Sharma K S , could you upload a branch-2 patch? It's better to commit both of them together.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Committing...

          Show
          leftnoteasy Wangda Tan added a comment - Committing...
          Hide
          hadoopqa Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12726211/0004-YARN-3410.patch
          against trunk revision d573f09.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7386//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7386//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12726211/0004-YARN-3410.patch against trunk revision d573f09. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 4 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7386//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7386//console This message is automatically generated.
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          All tests failed with BindException.. Jenkins need to kick off again to get another report!!

          Show
          rohithsharma Rohith Sharma K S added a comment - All tests failed with BindException.. Jenkins need to kick off again to get another report!!
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12726211/0004-YARN-3410.patch
          against trunk revision c6b5203.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified test files.

          -1 javac. The applied patch generated 1207 javac compiler warnings (more than the trunk's current 1181 warnings).

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:

          org.apache.hadoop.yarn.server.resourcemanager.TestApplicationACLs
          org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesFairScheduler
          org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps
          org.apache.hadoop.yarn.server.resourcemanager.security.TestAMRMTokens
          org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestAllocationFileLoaderService
          org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerQueueACLs
          org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart
          org.apache.hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA
          org.apache.hadoop.yarn.server.resourcemanager.recovery.TestFSRMStateStore

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7381//testReport/
          Javac warnings: https://builds.apache.org/job/PreCommit-YARN-Build/7381//artifact/patchprocess/diffJavacWarnings.txt
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7381//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12726211/0004-YARN-3410.patch against trunk revision c6b5203. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 4 new or modified test files. -1 javac . The applied patch generated 1207 javac compiler warnings (more than the trunk's current 1181 warnings). +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: org.apache.hadoop.yarn.server.resourcemanager.TestApplicationACLs org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesFairScheduler org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps org.apache.hadoop.yarn.server.resourcemanager.security.TestAMRMTokens org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestAllocationFileLoaderService org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerQueueACLs org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart org.apache.hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA org.apache.hadoop.yarn.server.resourcemanager.recovery.TestFSRMStateStore Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7381//testReport/ Javac warnings: https://builds.apache.org/job/PreCommit-YARN-Build/7381//artifact/patchprocess/diffJavacWarnings.txt Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7381//console This message is automatically generated.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Yes, in the same user two RM can not be started. It check for PID and fail it. YARN-2268 disallows the formatting state store while RM is running. The same verification can be made for this also in that JIRA

          Yes we should, it's the same problem.

          The latest patch LGTM, +1.

          Show
          leftnoteasy Wangda Tan added a comment - Yes, in the same user two RM can not be started. It check for PID and fail it. YARN-2268 disallows the formatting state store while RM is running. The same verification can be made for this also in that JIRA Yes we should, it's the same problem. The latest patch LGTM, +1.
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          Updated the patch fixing usage format.. kindly review updated patch

          Show
          rohithsharma Rohith Sharma K S added a comment - Updated the patch fixing usage format.. kindly review updated patch
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12726185/0003-YARN-3410.patch
          against trunk revision 76e7264.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:

          org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7378//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7378//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12726185/0003-YARN-3410.patch against trunk revision 76e7264. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 4 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7378//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7378//console This message is automatically generated.
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          I think RM will check pid while start to avoid this case, correct?

          Yes, in the same user two RM can not be started. It check for PID and fail it. YARN-2268 disallows the formatting state store while RM is running. The same verification can be made for this also in that JIRA

          Show
          rohithsharma Rohith Sharma K S added a comment - I think RM will check pid while start to avoid this case, correct? Yes, in the same user two RM can not be started. It check for PID and fail it. YARN-2268 disallows the formatting state store while RM is running. The same verification can be made for this also in that JIRA
          Hide
          leftnoteasy Wangda Tan added a comment -

          One question:
          What will happen if a running app is removed from state store while RM is running, will it cause state corrupted? I think RM will check pid while start to avoid this case, correct?

          And tried to deploy a local cluster to try this, everything works fine, one minor comment about usage:

          Usage: java ResourceManager [-format-state-store] |
          
             [-remove-application-from-state-store <ApplicationId>]
          

          Better to format it to?

          Usage: yarn resourcemanager [-format-state-store]
                                      [-remove..] <appId>
          
          Show
          leftnoteasy Wangda Tan added a comment - One question: What will happen if a running app is removed from state store while RM is running, will it cause state corrupted? I think RM will check pid while start to avoid this case, correct? And tried to deploy a local cluster to try this, everything works fine, one minor comment about usage: Usage: java ResourceManager [-format-state-store] | [-remove-application-from-state-store <ApplicationId>] Better to format it to? Usage: yarn resourcemanager [-format-state-store] [-remove..] <appId>
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          Thanks Wangda Tan for review.. Updated the patch that validated args as per review comment. Kindly review the updated the patch.

          Show
          rohithsharma Rohith Sharma K S added a comment - Thanks Wangda Tan for review.. Updated the patch that validated args as per review comment. Kindly review the updated the patch.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Patch generally looks good, but I suggest to improve the parameter checking a little:

          When args.length >=1, if args[0] == remove-app, check if args.length == 2, user can get more precise error about args not matching.

          Show
          leftnoteasy Wangda Tan added a comment - Patch generally looks good, but I suggest to improve the parameter checking a little: When args.length >=1, if args [0] == remove-app, check if args.length == 2, user can get more precise error about args not matching.
          Hide
          hadoopqa Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12724617/0002-YARN-3410.patch
          against trunk revision e516706.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7302//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7302//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12724617/0002-YARN-3410.patch against trunk revision e516706. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 4 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7302//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7302//console This message is automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12724602/0001-YARN-3410.patch
          against trunk revision 577d755.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:

          org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7300//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7300//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12724602/0001-YARN-3410.patch against trunk revision 577d755. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 4 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7300//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7300//console This message is automatically generated.
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          You need check argv[0] first, and then check argv.length == 2, "&&" will prevent get precise error message.

          First check should be done for argv.length because in normal RM start argue[0] is empty. So ArrayIndexOutOfBoundsException would be thrown

          Show
          rohithsharma Rohith Sharma K S added a comment - You need check argv [0] first, and then check argv.length == 2, "&&" will prevent get precise error message. First check should be done for argv.length because in normal RM start argue [0] is empty. So ArrayIndexOutOfBoundsException would be thrown
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          Updating the patch for fixing compilation problem.

          Show
          rohithsharma Rohith Sharma K S added a comment - Updating the patch for fixing compilation problem.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12724599/0001-YARN-3410.patch
          against trunk revision 577d755.

          -1 patch. Trunk compilation may be broken.

          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7299//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12724599/0001-YARN-3410.patch against trunk revision 577d755. -1 patch . Trunk compilation may be broken. Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7299//console This message is automatically generated.
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          And could you try to run the patch in a local cluster to see if CLI works?

          Yes, I have verified for ZKRMStateStore.

          Attached the updated patch fixing review comments. Kindly review that updated patch.

          Show
          rohithsharma Rohith Sharma K S added a comment - And could you try to run the patch in a local cluster to see if CLI works? Yes, I have verified for ZKRMStateStore. Attached the updated patch fixing review comments. Kindly review that updated patch.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Hi Rohith Sharma K S,
          Patch generally looks good, two minor comments:
          1) echo "...Use -remove-application-from-state-store for removing" add <appId> after -remove-application-from-state-store to tell user there's another arg?
          2)

          1203	      } else if (argv.length == 2
          1204	          && argv[0].equals("-remove-application-from-stare-store")) {
          

          You need check argv[0] first, and then check argv.length == 2, "&&" will prevent get precise error message.

          And could you try to run the patch in a local cluster to see if CLI works?

          Show
          leftnoteasy Wangda Tan added a comment - Hi Rohith Sharma K S , Patch generally looks good, two minor comments: 1) echo "...Use -remove-application-from-state-store for removing" add <appId> after -remove-application-from-state-store to tell user there's another arg? 2) 1203 } else if (argv.length == 2 1204 && argv[0].equals( "-remove-application-from-stare-store" )) { You need check argv [0] first, and then check argv.length == 2, "&&" will prevent get precise error message. And could you try to run the patch in a local cluster to see if CLI works?
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          Wangda Tan I'd appreciate if you have look at initial version of patch.

          Show
          rohithsharma Rohith Sharma K S added a comment - Wangda Tan I'd appreciate if you have look at initial version of patch.
          Hide
          hadoopqa Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12709200/0001-YARN-3410-v1.patch
          against trunk revision 96d7211.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7222//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7222//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12709200/0001-YARN-3410-v1.patch against trunk revision 96d7211. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 4 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7222//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7222//console This message is automatically generated.
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          Attached the initial patch for removing individual applications from state store.

          Show
          rohithsharma Rohith Sharma K S added a comment - Attached the initial patch for removing individual applications from state store.
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          what's the use case of using rmadmin removing a state while RM is running?

          Practically rmadmin need not to remove rm state store while RM running. I was thinking like if any exception happens during recovery like YARN-2340, then RM never get exited. RM keeps on switcing to standby and trying to become active. In this case, admin can format state store without stopping RM.

          it's better that RM can log all errors of applications recovering before die. With this, admin can know which application states caused RM die.

          I think this will be hard to get which application caused the problem ICO RuntimeExceptions. Admin need to back track the exception in the logs to identify it.

          Show
          rohithsharma Rohith Sharma K S added a comment - what's the use case of using rmadmin removing a state while RM is running? Practically rmadmin need not to remove rm state store while RM running. I was thinking like if any exception happens during recovery like YARN-2340 , then RM never get exited. RM keeps on switcing to standby and trying to become active. In this case, admin can format state store without stopping RM. it's better that RM can log all errors of applications recovering before die. With this, admin can know which application states caused RM die. I think this will be hard to get which application caused the problem ICO RuntimeExceptions. Admin need to back track the exception in the logs to identify it.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Thanks for your comment, Rohith Sharma K S.

          But what's the use case of using rmadmin removing a state while RM is running? The command is just a way to avoid app entered an un-expected state so RM cannot get started, unless there's any use case of doing that, I suggest to scope this to a RM starting option like YARN-2131.

          Show
          leftnoteasy Wangda Tan added a comment - Thanks for your comment, Rohith Sharma K S . But what's the use case of using rmadmin removing a state while RM is running? The command is just a way to avoid app entered an un-expected state so RM cannot get started, unless there's any use case of doing that, I suggest to scope this to a RM starting option like YARN-2131 .
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          For state store format in YARN-2131, discussion happened whether to format state using admin service or resourcemanager start up options comment link. Similarly I am thinking for application state deletion options

          1. ./yarn resourcemanager -delete-from-state-store app-id OR
          2. ./yarn rmadmin -delete-from-state-store app-id
            1st choice is pretty staight forward deletion neverthless of app state is finished or running. I would like to choose 2nd option.
          Show
          rohithsharma Rohith Sharma K S added a comment - For state store format in YARN-2131 , discussion happened whether to format state using admin service or resourcemanager start up options comment link . Similarly I am thinking for application state deletion options ./yarn resourcemanager -delete-from-state-store app-id OR ./yarn rmadmin -delete-from-state-store app-id 1st choice is pretty staight forward deletion neverthless of app state is finished or running. I would like to choose 2nd option.
          Hide
          leftnoteasy Wangda Tan added a comment -

          I think there is choice between start up option vs admin support. If both are in sync, then it would be better.

          Sorry I may not understand, this (remove individual application records) should be a command line running before RM get started. Could you elaborate this?

          Show
          leftnoteasy Wangda Tan added a comment - I think there is choice between start up option vs admin support. If both are in sync, then it would be better. Sorry I may not understand, this (remove individual application records) should be a command line running before RM get started. Could you elaborate this?
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          Just like YARN-2131 is handled , I think there is choice between start up option vs admin support. If both are in sync, then it would be better.

          Show
          rohithsharma Rohith Sharma K S added a comment - Just like YARN-2131 is handled , I think there is choice between start up option vs admin support. If both are in sync, then it would be better.
          Hide
          leftnoteasy Wangda Tan added a comment -

          It should be a improvement, thanks for taking this.

          Show
          leftnoteasy Wangda Tan added a comment - It should be a improvement, thanks for taking this.
          Hide
          leftnoteasy Wangda Tan added a comment -

          In console should be fine, what I meant is, it's better that RM can log all errors of applications recovering before die. With this, admin can know which application states caused RM die.

          Show
          leftnoteasy Wangda Tan added a comment - In console should be fine, what I meant is, it's better that RM can log all errors of applications recovering before die. With this, admin can know which application states caused RM die.
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          Wangda Tan IIUC, this is an Improvement on admin service. I have changed to Improvement. If you still think it is a Bug, let handle as Bug.

          Show
          rohithsharma Rohith Sharma K S added a comment - Wangda Tan IIUC, this is an Improvement on admin service. I have changed to Improvement. If you still think it is a Bug, let handle as Bug.
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          Giving privilege to admin is good option for removing single application entry from state store. There would be some other configurations which effect RM upon restart , may be need to revisit or identifies those configs.

          RM should be able to report all fatal errors (which will shutdown RM) when doing app recovery, this can save admin some time to remove apps in bad state.

          To be more clear, do you mean fatal error should be logged in logs or in console?

          Show
          rohithsharma Rohith Sharma K S added a comment - Giving privilege to admin is good option for removing single application entry from state store. There would be some other configurations which effect RM upon restart , may be need to revisit or identifies those configs. RM should be able to report all fatal errors (which will shutdown RM) when doing app recovery, this can save admin some time to remove apps in bad state. To be more clear, do you mean fatal error should be logged in logs or in console?

            People

            • Assignee:
              rohithsharma Rohith Sharma K S
              Reporter:
              leftnoteasy Wangda Tan
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development