Hadoop YARN
  1. Hadoop YARN
  2. YARN-1530

[Umbrella] Store, manage and serve per-framework application-timeline data

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      This is a sibling JIRA for YARN-321.

      Today, each application/framework has to do store, and serve per-framework data all by itself as YARN doesn't have a common solution. This JIRA attempts to solve the storage, management and serving of per-framework data from various applications, both running and finished. The aim is to change YARN to collect and store data in a generic manner with plugin points for frameworks to do their own thing w.r.t interpretation and serving.

      1. ATS-Write-Pipeline-Design-Proposal.pdf
        71 kB
        Robert Kanter
      2. ATS-meet-up-8-28-2014-notes.pdf
        80 kB
        Sangjin Lee
      3. application timeline design-20140210.pdf
        93 kB
        Billie Rinaldi
      4. application timeline design-20140130.pdf
        91 kB
        Billie Rinaldi
      5. application timeline design-20140116.pdf
        86 kB
        Billie Rinaldi
      6. application timeline design-20140108.pdf
        82 kB
        Vinod Kumar Vavilapalli

        Issue Links

        1.
        Define an in-memory implementation of ApplicationTimelineStore Sub-task Closed Zhijie Shen
         
        2.
        Define user-faced entity, entity-info and event objects Sub-task Closed Zhijie Shen
         
        3.
        Implement a Leveldb based ApplicationTimelineStore Sub-task Closed Billie Rinaldi
         
        4.
        Implement timeline related web-services inside AHS for storing and retrieving entities+events Sub-task Closed Zhijie Shen
         
        5.
        Implement a client library for java users to post entities+events Sub-task Closed Zhijie Shen
         
        6.
        Add an integration test validating post, storage and retrival of entites+events Sub-task Resolved Vinod Kumar Vavilapalli
         
        7.
        Define the ApplicationTimelineStore store as an abstraction for implementing different storage impls for storing timeline information Sub-task Closed Billie Rinaldi
         
        8.
        Refactoring timeline classes to remove "app" related words Sub-task Closed Zhijie Shen
         
        9.
        Rethinking about POJO Classes Sub-task Resolved Zhijie Shen
         
        10.
        Sending timeline entities+events from Distributed shell Sub-task Closed Mayank Bansal
         
        11.
        Replace MemoryApplicationTimelineStore with LeveldbApplicationTimelineStore as default Sub-task Closed Zhijie Shen
         
        12.
        Replace MemoryApplicationTimelineStore with LeveldbApplicationTimelineStore as default Sub-task Resolved Zhijie Shen
         
        13.
        Review LICENSE and NOTICE to reflect new levelDB releated libraries being used Sub-task Closed Billie Rinaldi
         
        14.
        Create an utility function to dump timeline records to json Sub-task Closed Zhijie Shen
         
        15.
        Enable offline deletion of entries in leveldb timeline store Sub-task Closed Billie Rinaldi
         
        16.
        ATSWebServices produces jersey warnings Sub-task Closed Billie Rinaldi
         
        17.
        TimelineWebServices always passes primary and secondary filters as strings Sub-task Closed Billie Rinaldi
         
        18.
        Leveldb timeline store needs simple write locking Sub-task Closed Billie Rinaldi
         
        19.
        Change types of related entities and primary filters in ATSEntity Sub-task Closed Billie Rinaldi
         
        20.
        Improve logging in timeline service Sub-task Closed Zhijie Shen
         
        21.
        Timeline service getEntities API should provide ability to get entities from given id Sub-task Closed Billie Rinaldi
         
        22.
        Make enabling timeline service configurable Sub-task Closed Zhijie Shen
         
        23.
        Improve default paths of timeline store and generic history store Sub-task Closed Tsuyoshi Ozawa
         
        24.
        Timeline server is enabled by default Sub-task Closed Mohammad Kamrul Islam
         
        25.
        LeveldbTimelineStore does not handle db exceptions properly Sub-task Closed Varun Saxena
         
        26.
        Implement a scalable, available TimelineStore using HBase Sub-task Resolved Li Lu
         
        27.
        Merging generic-history into the Timeline Store Sub-task Closed Zhijie Shen
         
        28.
        Rename the daemon name to timelineserver Sub-task Closed Zhijie Shen
         
        29.
        Rename internal names to being Timeline Service instead of application history Sub-task Resolved Naganarasimha G R
         
        30.
        Leveldb timeline store applies secondary filters incorrectly Sub-task Resolved Billie Rinaldi
         
        31.
        Add an admin module for the timeline server Sub-task Resolved Zhijie Shen
         
        32.
        DistributedShell publishes unfriendly user information to the timeline server Sub-task Closed Robert Kanter
         
        33.
        Refactor timeline classes into server.timeline package Sub-task Closed Vinod Kumar Vavilapalli
         
        34.
        Timeline server should validate the numeric configuration values Sub-task Closed Vasanth kumar RJ
         
        35.
        Timelineserver should validate that yarn.timeline-service.leveldb-timeline-store.ttl-interval-ms is greater than zero when level db is for timeline store Sub-task Resolved Unassigned
         
        36.
        Timeline server web interfaces high-availablity and scalability Sub-task Resolved Zhijie Shen
         
        37.
        Add Cross-Origin support to the ATS REST API Sub-task Closed Jonathan Eagles
         
        38.
        Move TimelineClient to yarn-common project Sub-task Closed Zhijie Shen
         
        39.
        Refactor TimelineWebServices Sub-task Closed Zhijie Shen
         
        40.
        Provide Cli to refesh Admin Acls for Timeline server Sub-task Resolved Varun Saxena
         
        41.
        Allow enabling/disabling timeline server per framework Sub-task Closed Mit Desai
         
        42.
        Rethinking of the HTTP method of TimelineWebServices#postEntities Sub-task Resolved Zhijie Shen
         
        43. TimelineClient should wrap all GET APIs to facilitate Java users Sub-task Patch Available Robert Kanter
         
        44.
        Document Cross Origin Filter Configuration for ATS Sub-task Closed Jonathan Eagles
         
        45.
        Cross Origin configuration parameters prefix are not honored Sub-task Closed Mit Desai
         
        46.
        Enable Cross Origin Filter for timeline server only and not all Yarn servers Sub-task Closed Mit Desai
         
        47.
        Allow All Origins by default when Cross Origin Filter is enabled Sub-task Closed Jonathan Eagles
         
        48.
        Allow for origin pattern matching in cross origin filter Sub-task Closed Jonathan Eagles
         
        49. Host framework UIs in YARN for use with the ATS Sub-task Open Jonathan Eagles
         
        50.
        Implement TimelineClientAsync Sub-task Resolved Tsuyoshi Ozawa
         
        51.
        Scalable and High Available Timeline Server Sub-task Resolved Zhijie Shen
         
        52.
        Reliable TimelineClient Sub-task Resolved Zhijie Shen
         
        53.
        Cross Origin Filter Http response split vulnerability protection rejects valid origins Sub-task Closed Jonathan Eagles
         
        54.
        Cross Origin Filter throws UnsupportedOperationException upon destroy Sub-task Closed Mit Desai
         
        55. Tool to measure the performance of the timeline server Sub-task Patch Available Chang Li
         
        56.
        Document of timeline server needs to be updated Sub-task Resolved Unassigned
         
        57.
        Add retry for timeline client put APIs Sub-task Closed Li Lu
         
        58.
        Add retry for timeline client getDelegationToken method Sub-task Closed Li Lu
         
        59.
        RM uses conf instead of token service address to renew timeline delegation tokens Sub-task Closed Jonathan Eagles
         
        60.
        Add Rolling Time To Lives Level DB Plugin Capabilities Sub-task Resolved Jonathan Eagles
         
        61.
        Add version info on timeline service / generic history web UI and REST API Sub-task Resolved Zhijie Shen
         

          Activity

          No work has yet been logged on this issue.

            People

            • Assignee:
              Unassigned
              Reporter:
              Vinod Kumar Vavilapalli
            • Votes:
              0 Vote for this issue
              Watchers:
              70 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development