Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3332

[Umbrella] Unified Resource Statistics Collection per node

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Today in YARN, NodeManager collects statistics like per container resource usage and overall physical resources available on the machine. Currently this is used internally in YARN by the NodeManager for only a limited usage: automatically determining the capacity of resources on node and enforcing memory usage to what is reserved per container.

      This proposal is to extend the existing architecture and collect statistics for usage b​eyond​ the existing use­cases.

      Proposal attached in comments.

        Attachments

        1. Design - UnifiedResourceStatisticsCollection.pdf
          894 kB
          Vinod Kumar Vavilapalli

          Issue Links

            Activity

              People

              • Assignee:
                vinodkv Vinod Kumar Vavilapalli
                Reporter:
                vinodkv Vinod Kumar Vavilapalli
              • Votes:
                0 Vote for this issue
                Watchers:
                30 Start watching this issue

                Dates

                • Created:
                  Updated: