Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-1530

[Umbrella] Store, manage and serve per-framework application-timeline data

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      This is a sibling JIRA for YARN-321.

      Today, each application/framework has to do store, and serve per-framework data all by itself as YARN doesn't have a common solution. This JIRA attempts to solve the storage, management and serving of per-framework data from various applications, both running and finished. The aim is to change YARN to collect and store data in a generic manner with plugin points for frameworks to do their own thing w.r.t interpretation and serving.

        Attachments

        1. application timeline design-20140108.pdf
          82 kB
          Vinod Kumar Vavilapalli
        2. application timeline design-20140116.pdf
          86 kB
          Billie Rinaldi
        3. application timeline design-20140130.pdf
          91 kB
          Billie Rinaldi
        4. application timeline design-20140210.pdf
          93 kB
          Billie Rinaldi
        5. ATS-meet-up-8-28-2014-notes.pdf
          80 kB
          Sangjin Lee
        6. ATS-Write-Pipeline-Design-Proposal.pdf
          71 kB
          Robert Kanter

        Issue Links

        1.
        Define an in-memory implementation of ApplicationTimelineStore Sub-task Closed Zhijie Shen Actions
        2.
        Define user-faced entity, entity-info and event objects Sub-task Closed Zhijie Shen Actions
        3.
        Implement a Leveldb based ApplicationTimelineStore Sub-task Closed Billie Rinaldi Actions
        4.
        Implement timeline related web-services inside AHS for storing and retrieving entities+events Sub-task Closed Zhijie Shen Actions
        5.
        Implement a client library for java users to post entities+events Sub-task Closed Zhijie Shen Actions
        6.
        Add an integration test validating post, storage and retrival of entites+events Sub-task Resolved Vinod Kumar Vavilapalli Actions
        7.
        Define the ApplicationTimelineStore store as an abstraction for implementing different storage impls for storing timeline information Sub-task Closed Billie Rinaldi Actions
        8.
        Refactoring timeline classes to remove "app" related words Sub-task Closed Zhijie Shen Actions
        9.
        Rethinking about POJO Classes Sub-task Resolved Zhijie Shen Actions
        10.
        Sending timeline entities+events from Distributed shell Sub-task Closed Mayank Bansal Actions
        11.
        Replace MemoryApplicationTimelineStore with LeveldbApplicationTimelineStore as default Sub-task Closed Zhijie Shen Actions
        12.
        Replace MemoryApplicationTimelineStore with LeveldbApplicationTimelineStore as default Sub-task Resolved Zhijie Shen Actions
        13.
        Review LICENSE and NOTICE to reflect new levelDB releated libraries being used Sub-task Closed Billie Rinaldi Actions
        14.
        Create an utility function to dump timeline records to json Sub-task Closed Zhijie Shen Actions
        15.
        Enable offline deletion of entries in leveldb timeline store Sub-task Closed Billie Rinaldi Actions
        16.
        ATSWebServices produces jersey warnings Sub-task Closed Billie Rinaldi Actions
        17.
        TimelineWebServices always passes primary and secondary filters as strings Sub-task Closed Billie Rinaldi Actions
        18.
        Leveldb timeline store needs simple write locking Sub-task Closed Billie Rinaldi Actions
        19.
        Change types of related entities and primary filters in ATSEntity Sub-task Closed Billie Rinaldi Actions
        20.
        Improve logging in timeline service Sub-task Closed Zhijie Shen Actions
        21.
        Timeline service getEntities API should provide ability to get entities from given id Sub-task Closed Billie Rinaldi Actions
        22.
        Make enabling timeline service configurable Sub-task Closed Zhijie Shen Actions
        23.
        Improve default paths of timeline store and generic history store Sub-task Closed Tsuyoshi Ozawa Actions
        24.
        Timeline server is enabled by default Sub-task Closed Mohammad Islam Actions
        25.
        LeveldbTimelineStore does not handle db exceptions properly Sub-task Closed Varun Saxena Actions
        26.
        Implement a scalable, available TimelineStore using HBase Sub-task Resolved Li Lu Actions
        27.
        Merging generic-history into the Timeline Store Sub-task Closed Zhijie Shen Actions
        28.
        Rename the daemon name to timelineserver Sub-task Closed Zhijie Shen Actions
        29.
        Rename internal names to being Timeline Service instead of application history Sub-task Resolved Naganarasimha G R Actions
        30.
        Leveldb timeline store applies secondary filters incorrectly Sub-task Resolved Billie Rinaldi Actions
        31.
        Add an admin module for the timeline server Sub-task Resolved Zhijie Shen Actions
        32.
        DistributedShell publishes unfriendly user information to the timeline server Sub-task Closed Robert Kanter Actions
        33.
        Refactor timeline classes into server.timeline package Sub-task Closed Vinod Kumar Vavilapalli Actions
        34.
        Timeline server should validate the numeric configuration values Sub-task Closed Vasanth kumar RJ Actions
        35.
        Timelineserver should validate that yarn.timeline-service.leveldb-timeline-store.ttl-interval-ms is greater than zero when level db is for timeline store Sub-task Resolved Unassigned Actions
        36.
        Timeline server web interfaces high-availablity and scalability Sub-task Resolved Zhijie Shen Actions
        37.
        Add Cross-Origin support to the ATS REST API Sub-task Closed Jonathan Turner Eagles Actions
        38.
        Move TimelineClient to yarn-common project Sub-task Closed Zhijie Shen Actions
        39.
        Refactor TimelineWebServices Sub-task Closed Zhijie Shen Actions
        40.
        Provide Cli to refesh Admin Acls for Timeline server Sub-task Resolved Varun Saxena Actions
        41.
        Allow enabling/disabling timeline server per framework Sub-task Closed Mit Desai Actions
        42.
        Rethinking of the HTTP method of TimelineWebServices#postEntities Sub-task Resolved Zhijie Shen Actions
        43.
        TimelineClient should wrap all GET APIs to facilitate Java users Sub-task Resolved Robert Kanter Actions
        44.
        Document Cross Origin Filter Configuration for ATS Sub-task Closed Jonathan Turner Eagles Actions
        45.
        Cross Origin configuration parameters prefix are not honored Sub-task Closed Mit Desai Actions
        46.
        Enable Cross Origin Filter for timeline server only and not all Yarn servers Sub-task Closed Mit Desai Actions
        47.
        Allow All Origins by default when Cross Origin Filter is enabled Sub-task Closed Jonathan Turner Eagles Actions
        48.
        Allow for origin pattern matching in cross origin filter Sub-task Closed Jonathan Turner Eagles Actions
        49.
        Host framework UIs in YARN for use with the ATS Sub-task Closed Jonathan Turner Eagles Actions
        50.
        Implement TimelineClientAsync Sub-task Resolved Tsuyoshi Ozawa Actions
        51.
        Scalable and High Available Timeline Server Sub-task Resolved Zhijie Shen Actions
        52.
        Reliable TimelineClient Sub-task Resolved Zhijie Shen Actions
        53.
        Cross Origin Filter Http response split vulnerability protection rejects valid origins Sub-task Closed Jonathan Turner Eagles Actions
        54.
        Cross Origin Filter throws UnsupportedOperationException upon destroy Sub-task Closed Mit Desai Actions
        55.
        Tool to measure the performance of the timeline server Sub-task Resolved Chang Li Actions
        56.
        Document of timeline server needs to be updated Sub-task Resolved Unassigned Actions
        57.
        Add retry for timeline client put APIs Sub-task Closed Li Lu Actions
        58.
        Add retry for timeline client getDelegationToken method Sub-task Closed Li Lu Actions
        59.
        RM uses conf instead of token service address to renew timeline delegation tokens Sub-task Closed Jonathan Turner Eagles Actions
        60.
        Add Rolling Time To Lives Level DB Plugin Capabilities Sub-task Resolved Jonathan Turner Eagles Actions
        61.
        Add version info on timeline service / generic history web UI and REST API Sub-task Resolved Zhijie Shen Actions

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              vinodkv Vinod Kumar Vavilapalli

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment