Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-896

Roll up for long-lived services in YARN

Add voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • None

    Description

      YARN is intended to be general purpose, but it is missing some features to be able to truly support long lived applications and long lived containers.

      This ticket is intended to

      1. discuss what is needed to support long lived processes
      2. track the resulting JIRA.

      Attachments

        Issue Links

        contains

        New Feature - A new feature of the product, which has yet to be developed. HADOOP-6959 Provisioning of long running Services via HOD

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Resolved
        depends upon

        Sub-task - The sub-task of the issue YARN-173 Page navigation support for container logs page and the logs web-service on NMs

        • Major - Major loss of function.
        • Open

        Sub-task - The sub-task of the issue YARN-624 Support gang scheduling in the AM RM protocol

        • Major - Major loss of function.
        • Open

        Sub-task - The sub-task of the issue YARN-1042 add ability to specify affinity/anti-affinity in container requests

        • Major - Major loss of function.
        • Open

        Sub-task - The sub-task of the issue YARN-1394 RM to inform AMs when a container completed due to NM going offline -planned or unplanned

        • Major - Major loss of function.
        • Resolved

        Sub-task - The sub-task of the issue YARN-611 Add an AM retry count reset window to YARN RM

        • Major - Major loss of function.
        • Closed

        Sub-task - The sub-task of the issue YARN-1111 NM containerlogs servlet can't handle logs of more than a GB

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Closed

        Bug - A problem which impairs or prevents the functions of the product. YARN-2648 need mechanism for updating HDFS delegation tokens associated with container launch contexts

        • Major - Major loss of function.
        • Open

        Bug - A problem which impairs or prevents the functions of the product. YARN-1489 [Umbrella] Work-preserving ApplicationMaster restart

        • Major - Major loss of function.
        • Resolved

        New Feature - A new feature of the product, which has yet to be developed. YARN-913 Umbrella: Add a way to register long-lived services in a YARN cluster

        • Major - Major loss of function.
        • Open

        New Feature - A new feature of the product, which has yet to be developed. YARN-2084 YARN to support REST APIs in AMs

        • Major - Major loss of function.
        • Open

        New Feature - A new feature of the product, which has yet to be developed. YARN-5907 [Umbrella] [YARN-1042] add ability to specify affinity/anti-affinity in container requests

        • Major - Major loss of function.
        • Open

        New Feature - A new feature of the product, which has yet to be developed. YARN-1105 RM or YarnClient to notify AMs if resource requests cannot be satisfied

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Open

        New Feature - A new feature of the product, which has yet to be developed. YARN-326 Add multi-resource scheduling to the fair scheduler

        • Major - Major loss of function.
        • Closed

        Improvement - An improvement or enhancement to an existing feature or task. YARN-4080 Capacity planning for long running services on YARN

        • Major - Major loss of function.
        • Open

        Improvement - An improvement or enhancement to an existing feature or task. YARN-1160 allow admins to force app deployment on a specific host

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Open

        Improvement - An improvement or enhancement to an existing feature or task. YARN-1200 Provide a central view for rack topologies

        • Major - Major loss of function.
        • Resolved

        Improvement - An improvement or enhancement to an existing feature or task. YARN-2005 Blacklisting support for scheduling AMs

        • Major - Major loss of function.
        • Resolved

        Improvement - An improvement or enhancement to an existing feature or task. YARN-614 Separate AM failures from hardware failure or YARN error and do not count them to AM retry count

        • Major - Major loss of function.
        • Closed

        Task - A task that needs to be done. YARN-445 Ability to signal containers

        • Major - Major loss of function.
        • Open

        Task - A task that needs to be done. YARN-1197 Support changing resources of an allocated container

        • Major - Major loss of function.
        • Open
        is blocked by

        Sub-task - The sub-task of the issue YARN-796 Allow for (admin) labels on nodes and resource-requests

        • Major - Major loss of function.
        • Open

        Sub-task - The sub-task of the issue YARN-1041 Protocol changes for RM to bind and notify a restarted AM of existing containers

        • Major - Major loss of function.
        • Closed

        Bug - A problem which impairs or prevents the functions of the product. YARN-3417 AM to be able to exit with a request saying "restart me with these (possibly updated) resource requirements"

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Open

        Improvement - An improvement or enhancement to an existing feature or task. YARN-3418 AM to be able to set/update web URL and IPC ports post-registration

        • Major - Major loss of function.
        • Open
        is depended upon by

        New Feature - A new feature of the product, which has yet to be developed. YARN-4407 Support Resource oversubscription in YARN scheduler

        • Major - Major loss of function.
        • Resolved
        is related to

        Epic - Created by Jira Software - do not edit or delete. Issue type for a big user story that needs to be broken down. AMBARI-17353 First class support for YARN hosted services

        • Critical - Crashes, loss of data, severe memory leak.
        • Open

        Bug - A problem which impairs or prevents the functions of the product. YARN-3057 Need update apps' runnability when reloading allocation files for FairScheduler

        • Major - Major loss of function.
        • Resolved

        New Feature - A new feature of the product, which has yet to be developed. YARN-2139 [Umbrella] Support for Disk as a Resource in YARN

        • Major - Major loss of function.
        • Open

        New Feature - A new feature of the product, which has yet to be developed. YARN-2140 Add support for network IO isolation/scheduling for containers

        • Major - Major loss of function.
        • Open

        New Feature - A new feature of the product, which has yet to be developed. YARN-679 add an entry point that can start any Yarn service

        • Major - Major loss of function.
        • Resolved

        Improvement - An improvement or enhancement to an existing feature or task. YARN-1151 Ability to configure auxiliary services from HDFS-based JAR files

        • Major - Major loss of function.
        • Resolved

        Umbrella - An overarching type made of sub-tasks SLIDER-183 Long lived application support

        • Major - Major loss of function.
        • Resolved
        relates to

        New Feature - A new feature of the product, which has yet to be developed. YARN-5025 Container move (relocation) between nodes

        • Major - Major loss of function.
        • Open

        Improvement - An improvement or enhancement to an existing feature or task. YARN-810 Support CGroup ceiling enforcement on CPU

        • Major - Major loss of function.
        • Open
        requires

        Task - A task that needs to be done. YARN-2443 Handling logs of long-running services on YARN

        • Major - Major loss of function.
        • Open

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            revans2 Robert Joseph Evans
            Votes:
            14 Vote for this issue
            Watchers:
            139 Start watching this issue

            Dates

              Created:
              Updated:

              Slack

                Issue deployment