Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4900

Dynamic configuration for task slots on TT

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 1.1.1
    • None
    • resourcemanager, tasktracker
    • None

    Description

      The current Hadoop MRV1 resource management logic assumes per node slot number is static during the lifetime of the TT process. Allowing run-time configuration on per node slot will give us finer granularity of resource elasticity. This allows Hadoop workloads to coexist with other workloads on the same hardware efficiently, whether or not the environment is virtualized.
      For more background or design details of this effort, please refer proposal in HADOOP-9165.

      Attachments

        1. MAPREDUCE-4900-demo.patch
          15 kB
          Binglin Chang
        2. MAPREDUCE-4900-demo-with-JMX.patch
          16 kB
          Binglin Chang
        3. MAPREDUCE-4900-demo-with-JMX.v2.patch
          17 kB
          Binglin Chang
        4. MAPREDUCE-4944-4900.jmx.v3.patch
          31 kB
          Binglin Chang
        5. MAPREDUCE-4944-4900.jmx.v4.patch
          36 kB
          Binglin Chang

        Issue Links

          blocks

          Improvement - An improvement or enhancement to an existing feature or task. MAPREDUCE-5381 Support graceful decommission of tasktracker

          • Major - Major loss of function.
          • Resolved
          is related to

          New Feature - A new feature of the product, which has yet to be developed. YARN-291 [Umbrella] Dynamic resource configuration

          • Major - Major loss of function.
          • Open

          New Feature - A new feature of the product, which has yet to be developed. MAPREDUCE-4944 Backport YARN-40 to support listClusterNodes and printNodeStatus in command line tool

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open
          supercedes

          New Feature - A new feature of the product, which has yet to be developed. HADOOP-9165 Dynamic Resource/Slot Configuration on NM/TT

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              decster Binglin Chang
              junping_du Junping Du
              Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

                Created:
                Updated: