Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3050

YarnScheduler needs to expose Resource Usage Information

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.23.0, 2.0.0-alpha
    • Fix Version/s: 0.23.0
    • Component/s: mrv2, resourcemanager
    • Labels:
      None

      Description

      Before the recent refactor The nodes had information in them about how much resources they were using. This information is not hidden inside SchedulerNode. Similarly resource usage information about an application, or in aggregate is only available through the Scheduler and there is not interface to pull it out.

      We need to expose APIs to get Resource and Container information from the scheduler, in aggregate across the entire cluster, per application, per node, and ideally also per queue if applicable (although there are no JIRAs I am aware of that need this right now).

      1. MR-3050.txt
        90 kB
        Robert Joseph Evans
      2. MR-3050.txt
        91 kB
        Robert Joseph Evans
      3. MR-3050.txt
        91 kB
        Robert Joseph Evans
      4. MR-3050.txt
        91 kB
        Robert Joseph Evans
      5. MR-3050.txt
        27 kB
        Robert Joseph Evans
      6. MR-3050.txt
        26 kB
        Robert Joseph Evans
      7. MR-3050.txt
        25 kB
        Robert Joseph Evans

        Issue Links

          Activity

          Allen Wittenauer made changes -
          Affects Version/s 2.0.0-alpha [ 12320354 ]
          Affects Version/s 0.24.0 [ 12317654 ]
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Arun C Murthy made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Target Version/s 0.23.0, 0.24.0 [ 12315570, 12317654 ] 0.23.0 [ 12315570 ]
          Fix Version/s 0.24.0 [ 12317654 ]
          Resolution Fixed [ 1 ]
          Robert Joseph Evans made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Robert Joseph Evans made changes -
          Attachment MR-3050.txt [ 12497212 ]
          Robert Joseph Evans made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Robert Joseph Evans made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Robert Joseph Evans made changes -
          Attachment MR-3050.txt [ 12497178 ]
          Robert Joseph Evans made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Robert Joseph Evans made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Robert Joseph Evans made changes -
          Attachment MR-3050.txt [ 12497012 ]
          Mahadev konar made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Vinod Kumar Vavilapalli made changes -
          Component/s mrv2 [ 12314301 ]
          Robert Joseph Evans made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Robert Joseph Evans made changes -
          Attachment MR-3050.txt [ 12496697 ]
          Robert Joseph Evans made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Robert Joseph Evans made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Target Version/s 0.24.0, 0.23.0 [ 12317654, 12315570 ] 0.23.0, 0.24.0 [ 12315570, 12317654 ]
          Robert Joseph Evans made changes -
          Attachment MR-3050.txt [ 12496512 ]
          Robert Joseph Evans made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Target Version/s 0.24.0, 0.23.0 [ 12317654, 12315570 ] 0.23.0, 0.24.0 [ 12315570, 12317654 ]
          Robert Joseph Evans made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Target Version/s 0.24.0, 0.23.0 [ 12317654, 12315570 ] 0.23.0, 0.24.0 [ 12315570, 12317654 ]
          Robert Joseph Evans made changes -
          Attachment MR-3050.txt [ 12496151 ]
          Robert Joseph Evans made changes -
          Attachment MR-3050.txt [ 12495431 ]
          Ravi Prakash made changes -
          Link This issue blocks MAPREDUCE-2790 [ MAPREDUCE-2790 ]
          Robert Joseph Evans made changes -
          Link This issue blocks MAPREDUCE-2738 [ MAPREDUCE-2738 ]
          Robert Joseph Evans made changes -
          Field Original Value New Value
          Link This issue blocks MAPREDUCE-2789 [ MAPREDUCE-2789 ]
          Robert Joseph Evans created issue -

            People

            • Assignee:
              Robert Joseph Evans
              Reporter:
              Robert Joseph Evans
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development