Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12180

Move ResourceCalculatorPlugin from YARN to Common

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: util
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Some of the monitoring functions could be moved from YARN to Common for easier sharing

      1. HADOOP-12180.000.patch
        94 kB
        Chris Douglas
      2. HADOOP-12180.001.patch
        94 kB
        Chris Douglas
      3. HADOOP-12180.002.patch
        94 kB
        Chris Douglas
      4. HADOOP-12180.003.patch
        94 kB
        Chris Douglas
      5. HADOOP-12180.004.patch
        94 kB
        Chris Douglas

        Issue Links

          Activity

          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #250 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/250/)
          HADOOP-12180. Move ResourceCalculatorPlugin from YARN to Common. (Chris Douglas via kasha) (kasha: rev ac6048372a58b3a3b57cd5f2702b44a3d4667f3d)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/CpuTimeTracker.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/LinuxResourceCalculatorPlugin.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ResourceCalculatorPlugin.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/CpuTimeTracker.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestSysInfoLinux.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestResourceCalculatorProcessTree.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsResourceCalculatorPlugin.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfo.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ProcfsBasedProcessTree.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestWindowsResourceCalculatorPlugin.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestLinuxResourceCalculatorPlugin.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoWindows.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestSysInfoWindows.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoLinux.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #250 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/250/ ) HADOOP-12180 . Move ResourceCalculatorPlugin from YARN to Common. (Chris Douglas via kasha) (kasha: rev ac6048372a58b3a3b57cd5f2702b44a3d4667f3d) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/CpuTimeTracker.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/LinuxResourceCalculatorPlugin.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ResourceCalculatorPlugin.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/CpuTimeTracker.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestSysInfoLinux.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestResourceCalculatorProcessTree.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsResourceCalculatorPlugin.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfo.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ProcfsBasedProcessTree.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestWindowsResourceCalculatorPlugin.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestLinuxResourceCalculatorPlugin.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoWindows.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestSysInfoWindows.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoLinux.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #240 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/240/)
          HADOOP-12180. Move ResourceCalculatorPlugin from YARN to Common. (Chris Douglas via kasha) (kasha: rev ac6048372a58b3a3b57cd5f2702b44a3d4667f3d)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ResourceCalculatorPlugin.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/LinuxResourceCalculatorPlugin.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoWindows.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsResourceCalculatorPlugin.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/CpuTimeTracker.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestSysInfoWindows.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfo.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/CpuTimeTracker.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestResourceCalculatorProcessTree.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoLinux.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ProcfsBasedProcessTree.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestWindowsResourceCalculatorPlugin.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestSysInfoLinux.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestLinuxResourceCalculatorPlugin.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #240 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/240/ ) HADOOP-12180 . Move ResourceCalculatorPlugin from YARN to Common. (Chris Douglas via kasha) (kasha: rev ac6048372a58b3a3b57cd5f2702b44a3d4667f3d) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ResourceCalculatorPlugin.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/LinuxResourceCalculatorPlugin.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoWindows.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsResourceCalculatorPlugin.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/CpuTimeTracker.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestSysInfoWindows.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfo.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/CpuTimeTracker.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestResourceCalculatorProcessTree.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoLinux.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ProcfsBasedProcessTree.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestWindowsResourceCalculatorPlugin.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestSysInfoLinux.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestLinuxResourceCalculatorPlugin.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2198 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2198/)
          HADOOP-12180. Move ResourceCalculatorPlugin from YARN to Common. (Chris Douglas via kasha) (kasha: rev ac6048372a58b3a3b57cd5f2702b44a3d4667f3d)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ProcfsBasedProcessTree.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoLinux.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/LinuxResourceCalculatorPlugin.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfo.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ResourceCalculatorPlugin.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestWindowsResourceCalculatorPlugin.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/CpuTimeTracker.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestResourceCalculatorProcessTree.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestLinuxResourceCalculatorPlugin.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoWindows.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestSysInfoWindows.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/CpuTimeTracker.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestSysInfoLinux.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsResourceCalculatorPlugin.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2198 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2198/ ) HADOOP-12180 . Move ResourceCalculatorPlugin from YARN to Common. (Chris Douglas via kasha) (kasha: rev ac6048372a58b3a3b57cd5f2702b44a3d4667f3d) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ProcfsBasedProcessTree.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoLinux.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/LinuxResourceCalculatorPlugin.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfo.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ResourceCalculatorPlugin.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestWindowsResourceCalculatorPlugin.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/CpuTimeTracker.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestResourceCalculatorProcessTree.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestLinuxResourceCalculatorPlugin.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoWindows.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestSysInfoWindows.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/CpuTimeTracker.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestSysInfoLinux.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsResourceCalculatorPlugin.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2179 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2179/)
          HADOOP-12180. Move ResourceCalculatorPlugin from YARN to Common. (Chris Douglas via kasha) (kasha: rev ac6048372a58b3a3b57cd5f2702b44a3d4667f3d)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestResourceCalculatorProcessTree.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoWindows.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/LinuxResourceCalculatorPlugin.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoLinux.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ProcfsBasedProcessTree.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/CpuTimeTracker.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsResourceCalculatorPlugin.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfo.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestWindowsResourceCalculatorPlugin.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/CpuTimeTracker.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ResourceCalculatorPlugin.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestSysInfoWindows.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestLinuxResourceCalculatorPlugin.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestSysInfoLinux.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2179 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2179/ ) HADOOP-12180 . Move ResourceCalculatorPlugin from YARN to Common. (Chris Douglas via kasha) (kasha: rev ac6048372a58b3a3b57cd5f2702b44a3d4667f3d) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestResourceCalculatorProcessTree.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoWindows.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/LinuxResourceCalculatorPlugin.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoLinux.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ProcfsBasedProcessTree.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/CpuTimeTracker.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsResourceCalculatorPlugin.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfo.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestWindowsResourceCalculatorPlugin.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/CpuTimeTracker.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ResourceCalculatorPlugin.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestSysInfoWindows.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestLinuxResourceCalculatorPlugin.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestSysInfoLinux.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #982 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/982/)
          HADOOP-12180. Move ResourceCalculatorPlugin from YARN to Common. (Chris Douglas via kasha) (kasha: rev ac6048372a58b3a3b57cd5f2702b44a3d4667f3d)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestLinuxResourceCalculatorPlugin.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestWindowsResourceCalculatorPlugin.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ResourceCalculatorPlugin.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfo.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/CpuTimeTracker.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/CpuTimeTracker.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/LinuxResourceCalculatorPlugin.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ProcfsBasedProcessTree.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestResourceCalculatorProcessTree.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoWindows.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsResourceCalculatorPlugin.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestSysInfoLinux.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestSysInfoWindows.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoLinux.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #982 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/982/ ) HADOOP-12180 . Move ResourceCalculatorPlugin from YARN to Common. (Chris Douglas via kasha) (kasha: rev ac6048372a58b3a3b57cd5f2702b44a3d4667f3d) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestLinuxResourceCalculatorPlugin.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestWindowsResourceCalculatorPlugin.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ResourceCalculatorPlugin.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfo.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/CpuTimeTracker.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/CpuTimeTracker.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/LinuxResourceCalculatorPlugin.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ProcfsBasedProcessTree.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestResourceCalculatorProcessTree.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoWindows.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsResourceCalculatorPlugin.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestSysInfoLinux.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestSysInfoWindows.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoLinux.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #252 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/252/)
          HADOOP-12180. Move ResourceCalculatorPlugin from YARN to Common. (Chris Douglas via kasha) (kasha: rev ac6048372a58b3a3b57cd5f2702b44a3d4667f3d)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoLinux.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/CpuTimeTracker.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestSysInfoWindows.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/CpuTimeTracker.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsResourceCalculatorPlugin.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfo.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestLinuxResourceCalculatorPlugin.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ResourceCalculatorPlugin.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestWindowsResourceCalculatorPlugin.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ProcfsBasedProcessTree.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoWindows.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestSysInfoLinux.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestResourceCalculatorProcessTree.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/LinuxResourceCalculatorPlugin.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #252 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/252/ ) HADOOP-12180 . Move ResourceCalculatorPlugin from YARN to Common. (Chris Douglas via kasha) (kasha: rev ac6048372a58b3a3b57cd5f2702b44a3d4667f3d) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoLinux.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/CpuTimeTracker.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestSysInfoWindows.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/CpuTimeTracker.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsResourceCalculatorPlugin.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfo.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestLinuxResourceCalculatorPlugin.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ResourceCalculatorPlugin.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestWindowsResourceCalculatorPlugin.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ProcfsBasedProcessTree.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoWindows.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestSysInfoLinux.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestResourceCalculatorProcessTree.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/LinuxResourceCalculatorPlugin.java
          Hide
          chris.douglas Chris Douglas added a comment -

          Thanks, Karthik

          Show
          chris.douglas Chris Douglas added a comment - Thanks, Karthik
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8141 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8141/)
          HADOOP-12180. Move ResourceCalculatorPlugin from YARN to Common. (Chris Douglas via kasha) (kasha: rev ac6048372a58b3a3b57cd5f2702b44a3d4667f3d)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoWindows.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/CpuTimeTracker.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ProcfsBasedProcessTree.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsResourceCalculatorPlugin.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/CpuTimeTracker.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestLinuxResourceCalculatorPlugin.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoLinux.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestSysInfoLinux.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestResourceCalculatorProcessTree.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/LinuxResourceCalculatorPlugin.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ResourceCalculatorPlugin.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestWindowsResourceCalculatorPlugin.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfo.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestSysInfoWindows.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8141 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8141/ ) HADOOP-12180 . Move ResourceCalculatorPlugin from YARN to Common. (Chris Douglas via kasha) (kasha: rev ac6048372a58b3a3b57cd5f2702b44a3d4667f3d) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoWindows.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/CpuTimeTracker.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ProcfsBasedProcessTree.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsResourceCalculatorPlugin.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/CpuTimeTracker.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestLinuxResourceCalculatorPlugin.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoLinux.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestSysInfoLinux.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestResourceCalculatorProcessTree.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/LinuxResourceCalculatorPlugin.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ResourceCalculatorPlugin.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestWindowsResourceCalculatorPlugin.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfo.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestSysInfoWindows.java
          Hide
          kasha Karthik Kambatla added a comment -

          Thanks Chris for working on this. Just committed to trunk and branch-2.

          Show
          kasha Karthik Kambatla added a comment - Thanks Chris for working on this. Just committed to trunk and branch-2.
          Hide
          kasha Karthik Kambatla added a comment -

          +1. Checking this in.

          Show
          kasha Karthik Kambatla added a comment - +1. Checking this in.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 17m 14s Findbugs (version ) appears to be broken on trunk.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 5 new or modified test files.
          +1 javac 7m 49s There were no new javac warning messages.
          +1 javadoc 9m 50s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 1m 31s There were no new checkstyle issues.
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 19s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 3m 23s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 common tests 22m 10s Tests passed in hadoop-common.
          +1 yarn tests 1m 55s Tests passed in hadoop-yarn-common.
              66m 13s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12744345/HADOOP-12180.004.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 2e3d83f
          hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7205/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7205/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7205/testReport/
          Java 1.7.0_55
          uname Linux asf908.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7205/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 17m 14s Findbugs (version ) appears to be broken on trunk. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 5 new or modified test files. +1 javac 7m 49s There were no new javac warning messages. +1 javadoc 9m 50s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 31s There were no new checkstyle issues. +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 19s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 3m 23s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 common tests 22m 10s Tests passed in hadoop-common. +1 yarn tests 1m 55s Tests passed in hadoop-yarn-common.     66m 13s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12744345/HADOOP-12180.004.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 2e3d83f hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7205/artifact/patchprocess/testrun_hadoop-common.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7205/artifact/patchprocess/testrun_hadoop-yarn-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7205/testReport/ Java 1.7.0_55 uname Linux asf908.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7205/console This message was automatically generated.
          Hide
          chris.douglas Chris Douglas added a comment -

          Sounds good; replaced w/ UnsupportedOperationException

          Show
          chris.douglas Chris Douglas added a comment - Sounds good; replaced w/ UnsupportedOperationException
          Hide
          kasha Karthik Kambatla added a comment -

          What do you think of UnsupportedOperationException or a custom UnknownOSException?

          Show
          kasha Karthik Kambatla added a comment - What do you think of UnsupportedOperationException or a custom UnknownOSException ?
          Hide
          chris.douglas Chris Douglas added a comment -

          SysInfo#newInstance - if OS type is not Linux or Windows, is that a security issue? Would RuntimeException be more appropriate than SecurityException?

          It throws SecurityException because RCP::getResourceCalculatorPlugin caught it when creating the impl fails, not for any particularly good reason. I'm happy to replace it with something else including RuntimeException, though ideally there's a more specific type to catch at RCP to avoid swallowing exceptions there.

          Show
          chris.douglas Chris Douglas added a comment - SysInfo#newInstance - if OS type is not Linux or Windows, is that a security issue? Would RuntimeException be more appropriate than SecurityException? It throws SecurityException because RCP::getResourceCalculatorPlugin caught it when creating the impl fails, not for any particularly good reason. I'm happy to replace it with something else including RuntimeException , though ideally there's a more specific type to catch at RCP to avoid swallowing exceptions there.
          Hide
          kasha Karthik Kambatla added a comment -

          Patch looks mostly good, just one question.

          SysInfo#newInstance - if OS type is not Linux or Windows, is that a security issue? Would RuntimeException be more appropriate than SecurityException?

          Show
          kasha Karthik Kambatla added a comment - Patch looks mostly good, just one question. SysInfo#newInstance - if OS type is not Linux or Windows, is that a security issue? Would RuntimeException be more appropriate than SecurityException?
          Hide
          chris.douglas Chris Douglas added a comment -

          Soright, I'll do that. Can you take a look at the patch?

          Show
          chris.douglas Chris Douglas added a comment - Soright, I'll do that. Can you take a look at the patch?
          Hide
          kasha Karthik Kambatla added a comment -

          For YARN-3332 and other related work, it would be simpler if trunk and branch-2 are same.

          Show
          kasha Karthik Kambatla added a comment - For YARN-3332 and other related work, it would be simpler if trunk and branch-2 are same.
          Hide
          chris.douglas Chris Douglas added a comment -

          Any reason we are targeting only trunk and not branch-2?

          Nope. I can backport if it'd be useful.

          Show
          chris.douglas Chris Douglas added a comment - Any reason we are targeting only trunk and not branch-2? Nope. I can backport if it'd be useful.
          Hide
          kasha Karthik Kambatla added a comment -

          Any reason we are targeting only trunk and not branch-2?

          Show
          kasha Karthik Kambatla added a comment - Any reason we are targeting only trunk and not branch-2?
          Hide
          hadoopqa Hadoop QA added a comment -



          +1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 18m 57s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 5 new or modified test files.
          +1 javac 7m 44s There were no new javac warning messages.
          +1 javadoc 9m 40s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 1m 43s There were no new checkstyle issues.
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 33s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 3m 20s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 common tests 22m 24s Tests passed in hadoop-common.
          +1 yarn tests 1m 56s Tests passed in hadoop-yarn-common.
              68m 17s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12743853/HADOOP-12180.003.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 81f3644
          hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7174/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7174/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7174/testReport/
          Java 1.7.0_55
          uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7174/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 57s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 5 new or modified test files. +1 javac 7m 44s There were no new javac warning messages. +1 javadoc 9m 40s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 43s There were no new checkstyle issues. +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 3m 20s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 common tests 22m 24s Tests passed in hadoop-common. +1 yarn tests 1m 56s Tests passed in hadoop-yarn-common.     68m 17s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12743853/HADOOP-12180.003.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 81f3644 hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7174/artifact/patchprocess/testrun_hadoop-common.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7174/artifact/patchprocess/testrun_hadoop-yarn-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7174/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7174/console This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 19m 18s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 5 new or modified test files.
          +1 javac 8m 18s There were no new javac warning messages.
          +1 javadoc 10m 18s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 39s The applied patch generated 1 new checkstyle issues (total was 0, now 1).
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 40s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 3m 41s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          -1 common tests 22m 41s Tests failed in hadoop-common.
          +1 yarn tests 2m 3s Tests passed in hadoop-yarn-common.
              70m 56s  



          Reason Tests
          Failed unit tests hadoop.fs.TestSymlinkLocalFSFileSystem



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12743835/HADOOP-12180.002.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / bf89ddb
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/7170/artifact/patchprocess/diffcheckstylehadoop-common.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7170/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7170/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7170/testReport/
          Java 1.7.0_55
          uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7170/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 19m 18s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 5 new or modified test files. +1 javac 8m 18s There were no new javac warning messages. +1 javadoc 10m 18s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 39s The applied patch generated 1 new checkstyle issues (total was 0, now 1). +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 40s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 3m 41s The patch does not introduce any new Findbugs (version 3.0.0) warnings. -1 common tests 22m 41s Tests failed in hadoop-common. +1 yarn tests 2m 3s Tests passed in hadoop-yarn-common.     70m 56s   Reason Tests Failed unit tests hadoop.fs.TestSymlinkLocalFSFileSystem Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12743835/HADOOP-12180.002.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / bf89ddb checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/7170/artifact/patchprocess/diffcheckstylehadoop-common.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7170/artifact/patchprocess/testrun_hadoop-common.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7170/artifact/patchprocess/testrun_hadoop-yarn-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7170/testReport/ Java 1.7.0_55 uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7170/console This message was automatically generated.
          Hide
          chris.douglas Chris Douglas added a comment -

          Removed tabs from copied TestLinuxResourceCalculatorPlugin

          Show
          chris.douglas Chris Douglas added a comment - Removed tabs from copied TestLinuxResourceCalculatorPlugin
          Hide
          chris.douglas Chris Douglas added a comment -

          Soright; changed SysInfo to Public/Evolving, and the impls to Private/Evolving

          Show
          chris.douglas Chris Douglas added a comment - Soright; changed SysInfo to Public/Evolving, and the impls to Private/Evolving
          Hide
          aw Allen Wittenauer added a comment -

          Yes, please.

          LimitedPrivate is just kind of terrible.

          Show
          aw Allen Wittenauer added a comment - Yes, please. LimitedPrivate is just kind of terrible.
          Hide
          elgoiri Íñigo Goiri added a comment -

          Karthik Kambatla is usually my spiritual guide on these annotations but I'd say yest; that makes sense.

          Show
          elgoiri Íñigo Goiri added a comment - Karthik Kambatla is usually my spiritual guide on these annotations but I'd say yest; that makes sense.
          Hide
          chris.douglas Chris Douglas added a comment -

          Actually... how about marking SysInfo as Public/Evolving instead of LimitedPrivate/Unstable?

          Show
          chris.douglas Chris Douglas added a comment - Actually... how about marking SysInfo as Public/Evolving instead of LimitedPrivate/Unstable?
          Hide
          chris.douglas Chris Douglas added a comment -

          I think that the SysInfo name is not as meaningful as ResourceCalculator but I don't have better proposals so I guess is fine

          Unfortunately, YARN also has a o.a.h.yarn.util.resource.ResourceCalculator that's not related to o.a.h.yarn.util.ResourceCalculatorPlugin. I'm open to other names, but SysInfo seemed to cover the /proc scraping this performs.

          What about making ResourceCalculatorPlugin extend/implement SysInfo to make the dependency more clear? This would also change the initialization of the other plugins and may not work.

          Agreed, this is awkward. If RCP extends SysInfo, then the impls in YARN can't be related to the impls in Common. SysInfo as an interface doesn't really make anything cleaner; RCP would need to be an interface, and interfaces can't have static methods (i.e., RCP::getResourceCalculatorPlugin). Letting RCP be an (extensible) wrapper around SysInfo preserves existing RCP implementations in YARN. If we relax the backwards-compat we could do better, but the gains seem mostly aesthetic.

          I'll add the LimitedPrivate annotation. I'll file a separate JIRA for thread safety, so YARN-3819, YARN-3820 can merge and we can consider the whole picture.

          Show
          chris.douglas Chris Douglas added a comment - I think that the SysInfo name is not as meaningful as ResourceCalculator but I don't have better proposals so I guess is fine Unfortunately, YARN also has a o.a.h.yarn.util.resource.ResourceCalculator that's not related to o.a.h.yarn.util.ResourceCalculatorPlugin. I'm open to other names, but SysInfo seemed to cover the /proc scraping this performs. What about making ResourceCalculatorPlugin extend/implement SysInfo to make the dependency more clear? This would also change the initialization of the other plugins and may not work. Agreed, this is awkward. If RCP extends SysInfo, then the impls in YARN can't be related to the impls in Common. SysInfo as an interface doesn't really make anything cleaner; RCP would need to be an interface, and interfaces can't have static methods (i.e., RCP::getResourceCalculatorPlugin). Letting RCP be an (extensible) wrapper around SysInfo preserves existing RCP implementations in YARN. If we relax the backwards-compat we could do better, but the gains seem mostly aesthetic. I'll add the LimitedPrivate annotation. I'll file a separate JIRA for thread safety, so YARN-3819 , YARN-3820 can merge and we can consider the whole picture.
          Hide
          elgoiri Íñigo Goiri added a comment -
          • I think that the SysInfo name is not as meaningful as ResourceCalculator but I don't have better proposals so I guess is fine
          • Can we add HDFS to LimitedPrivate in SysInfo?
          • What about making ResourceCalculatorPlugin extend/implement SysInfo to make the dependency more clear? This would also change the initialization of the other plugins and may not work.
          • +1 on threadsafety, I think is worth doing; not sure if here or in a separate JIRA.
          Show
          elgoiri Íñigo Goiri added a comment - I think that the SysInfo name is not as meaningful as ResourceCalculator but I don't have better proposals so I guess is fine Can we add HDFS to LimitedPrivate in SysInfo? What about making ResourceCalculatorPlugin extend/implement SysInfo to make the dependency more clear? This would also change the initialization of the other plugins and may not work. +1 on threadsafety, I think is worth doing; not sure if here or in a separate JIRA.
          Hide
          hadoopqa Hadoop QA added a comment -



          +1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 17m 52s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 5 new or modified test files.
          +1 javac 7m 31s There were no new javac warning messages.
          +1 javadoc 9m 38s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 1m 41s There were no new checkstyle issues.
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 35s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 3m 24s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 common tests 22m 43s Tests passed in hadoop-common.
          +1 yarn tests 1m 54s Tests passed in hadoop-yarn-common.
              67m 20s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12743447/HADOOP-12180.001.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 2eae130
          hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7139/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7139/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7139/testReport/
          Java 1.7.0_55
          uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7139/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 52s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 5 new or modified test files. +1 javac 7m 31s There were no new javac warning messages. +1 javadoc 9m 38s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 41s There were no new checkstyle issues. +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 35s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 3m 24s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 common tests 22m 43s Tests passed in hadoop-common. +1 yarn tests 1m 54s Tests passed in hadoop-yarn-common.     67m 20s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12743447/HADOOP-12180.001.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 2eae130 hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7139/artifact/patchprocess/testrun_hadoop-common.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7139/artifact/patchprocess/testrun_hadoop-yarn-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7139/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7139/console This message was automatically generated.
          Hide
          chris.douglas Chris Douglas added a comment -

          Summary of changes:

          • Created o.a.h.util.SysInfo with the same interface as ResourceCalculatorPlugin
          • Changed ResourceCalculatorPlugin to delegate to a SysInfo instance, and the OS-specific RCP impls to pass the equivalent SysInfoLinux and SysInfoWindows classes to the (now concrete) parent class
          • CpuTimeTracker also got carried to Common to avoid a dep on YARN; resolved some references to the process tree utility classes
          • Fixed some of the unit tests and classes to satisfy findbugs/checkstyle rules, but the impls are essentially unchanged.

          This should allow the YARN ResourceCalculatorPlugin classes to be configurable as before, including those overriding the API methods of OS-specific impls (of course, any extension relying on impl details of the RCs will break). This does not implement comparable pluggability for SysInfo::newInstance, which only does the same OS check as in the current code. I also elected not to extend Configured or implement Configurable for SysInfo, as none of the impls use it. If SysInfo should check the config, use a ServiceLoader, or some other method for selecting the "correct" impl: that could be follow-on work.

          The SysInfoLinux impl (and extensions in YARN-3819, YARN-3820) all scrape /proc, so maybe SysInfoLinuxProc would be more apt (some applies to Windows impl), but absent other implementations and the prenominate config... I'm inclined to ignore it.

          These implementions aren't threadsafe. If we're moving them to Common (from internal classes for YARN) that may be worth fixing, but I wanted to see what others thought first. It could also be a separate JIRA.

          Show
          chris.douglas Chris Douglas added a comment - Summary of changes: Created o.a.h.util.SysInfo with the same interface as ResourceCalculatorPlugin Changed ResourceCalculatorPlugin to delegate to a SysInfo instance, and the OS-specific RCP impls to pass the equivalent SysInfoLinux and SysInfoWindows classes to the (now concrete) parent class CpuTimeTracker also got carried to Common to avoid a dep on YARN; resolved some references to the process tree utility classes Fixed some of the unit tests and classes to satisfy findbugs/checkstyle rules, but the impls are essentially unchanged. This should allow the YARN ResourceCalculatorPlugin classes to be configurable as before, including those overriding the API methods of OS-specific impls (of course, any extension relying on impl details of the RCs will break). This does not implement comparable pluggability for SysInfo::newInstance , which only does the same OS check as in the current code. I also elected not to extend Configured or implement Configurable for SysInfo , as none of the impls use it. If SysInfo should check the config, use a ServiceLoader , or some other method for selecting the "correct" impl: that could be follow-on work. The SysInfoLinux impl (and extensions in YARN-3819 , YARN-3820 ) all scrape /proc , so maybe SysInfoLinuxProc would be more apt (some applies to Windows impl), but absent other implementations and the prenominate config... I'm inclined to ignore it. These implementions aren't threadsafe. If we're moving them to Common (from internal classes for YARN) that may be worth fixing, but I wanted to see what others thought first. It could also be a separate JIRA.
          Hide
          chris.douglas Chris Douglas added a comment -

          Whitespace.

          Show
          chris.douglas Chris Douglas added a comment - Whitespace.

            People

            • Assignee:
              chris.douglas Chris Douglas
              Reporter:
              chris.douglas Chris Douglas
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development