Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-1530

[Umbrella] Store, manage and serve per-framework application-timeline data

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      This is a sibling JIRA for YARN-321.

      Today, each application/framework has to do store, and serve per-framework data all by itself as YARN doesn't have a common solution. This JIRA attempts to solve the storage, management and serving of per-framework data from various applications, both running and finished. The aim is to change YARN to collect and store data in a generic manner with plugin points for frameworks to do their own thing w.r.t interpretation and serving.

        Attachments

        1. application timeline design-20140108.pdf
          82 kB
          Vinod Kumar Vavilapalli
        2. application timeline design-20140116.pdf
          86 kB
          Billie Rinaldi
        3. application timeline design-20140130.pdf
          91 kB
          Billie Rinaldi
        4. application timeline design-20140210.pdf
          93 kB
          Billie Rinaldi
        5. ATS-meet-up-8-28-2014-notes.pdf
          80 kB
          Sangjin Lee
        6. ATS-Write-Pipeline-Design-Proposal.pdf
          71 kB
          Robert Kanter

          Issue Links

          1.
          Define an in-memory implementation of ApplicationTimelineStore Sub-task Closed Zhijie Shen
          2.
          Define user-faced entity, entity-info and event objects Sub-task Closed Zhijie Shen
          3.
          Implement a Leveldb based ApplicationTimelineStore Sub-task Closed Billie Rinaldi
          4.
          Implement timeline related web-services inside AHS for storing and retrieving entities+events Sub-task Closed Zhijie Shen
          5.
          Implement a client library for java users to post entities+events Sub-task Closed Zhijie Shen
          6.
          Add an integration test validating post, storage and retrival of entites+events Sub-task Resolved Vinod Kumar Vavilapalli
          7.
          Define the ApplicationTimelineStore store as an abstraction for implementing different storage impls for storing timeline information Sub-task Closed Billie Rinaldi
          8.
          Refactoring timeline classes to remove "app" related words Sub-task Closed Zhijie Shen
          9.
          Rethinking about POJO Classes Sub-task Resolved Zhijie Shen
          10.
          Sending timeline entities+events from Distributed shell Sub-task Closed Mayank Bansal
          11.
          Replace MemoryApplicationTimelineStore with LeveldbApplicationTimelineStore as default Sub-task Closed Zhijie Shen
          12.
          Replace MemoryApplicationTimelineStore with LeveldbApplicationTimelineStore as default Sub-task Resolved Zhijie Shen
          13.
          Review LICENSE and NOTICE to reflect new levelDB releated libraries being used Sub-task Closed Billie Rinaldi
          14.
          Create an utility function to dump timeline records to json Sub-task Closed Zhijie Shen
          15.
          Enable offline deletion of entries in leveldb timeline store Sub-task Closed Billie Rinaldi
          16.
          ATSWebServices produces jersey warnings Sub-task Closed Billie Rinaldi
          17.
          TimelineWebServices always passes primary and secondary filters as strings Sub-task Closed Billie Rinaldi
          18.
          Leveldb timeline store needs simple write locking Sub-task Closed Billie Rinaldi
          19.
          Change types of related entities and primary filters in ATSEntity Sub-task Closed Billie Rinaldi
          20.
          Improve logging in timeline service Sub-task Closed Zhijie Shen
          21.
          Timeline service getEntities API should provide ability to get entities from given id Sub-task Closed Billie Rinaldi
          22.
          Make enabling timeline service configurable Sub-task Closed Zhijie Shen
          23.
          Improve default paths of timeline store and generic history store Sub-task Closed Tsuyoshi Ozawa
          24.
          Timeline server is enabled by default Sub-task Closed Mohammad Kamrul Islam
          25.
          LeveldbTimelineStore does not handle db exceptions properly Sub-task Closed Varun Saxena
          26.
          Implement a scalable, available TimelineStore using HBase Sub-task Resolved Li Lu
          27.
          Merging generic-history into the Timeline Store Sub-task Closed Zhijie Shen
          28.
          Rename the daemon name to timelineserver Sub-task Closed Zhijie Shen
          29.
          Rename internal names to being Timeline Service instead of application history Sub-task Resolved Naganarasimha G R
          30.
          Leveldb timeline store applies secondary filters incorrectly Sub-task Resolved Billie Rinaldi
          31.
          Add an admin module for the timeline server Sub-task Resolved Zhijie Shen
          32.
          DistributedShell publishes unfriendly user information to the timeline server Sub-task Closed Robert Kanter
          33.
          Refactor timeline classes into server.timeline package Sub-task Closed Vinod Kumar Vavilapalli
          34.
          Timeline server should validate the numeric configuration values Sub-task Closed Vasanth kumar RJ
          35.
          Timelineserver should validate that yarn.timeline-service.leveldb-timeline-store.ttl-interval-ms is greater than zero when level db is for timeline store Sub-task Resolved Unassigned
          36.
          Timeline server web interfaces high-availablity and scalability Sub-task Resolved Zhijie Shen
          37.
          Add Cross-Origin support to the ATS REST API Sub-task Closed Jonathan Eagles
          38.
          Move TimelineClient to yarn-common project Sub-task Closed Zhijie Shen
          39.
          Refactor TimelineWebServices Sub-task Closed Zhijie Shen
          40.
          Provide Cli to refesh Admin Acls for Timeline server Sub-task Resolved Varun Saxena
          41.
          Allow enabling/disabling timeline server per framework Sub-task Closed Mit Desai
          42.
          Rethinking of the HTTP method of TimelineWebServices#postEntities Sub-task Resolved Zhijie Shen
          43.
          TimelineClient should wrap all GET APIs to facilitate Java users Sub-task Resolved Robert Kanter
          44.
          Document Cross Origin Filter Configuration for ATS Sub-task Closed Jonathan Eagles
          45.
          Cross Origin configuration parameters prefix are not honored Sub-task Closed Mit Desai
          46.
          Enable Cross Origin Filter for timeline server only and not all Yarn servers Sub-task Closed Mit Desai
          47.
          Allow All Origins by default when Cross Origin Filter is enabled Sub-task Closed Jonathan Eagles
          48.
          Allow for origin pattern matching in cross origin filter Sub-task Closed Jonathan Eagles
          49.
          Host framework UIs in YARN for use with the ATS Sub-task Closed Jonathan Eagles
          50.
          Implement TimelineClientAsync Sub-task Resolved Tsuyoshi Ozawa
          51.
          Scalable and High Available Timeline Server Sub-task Resolved Zhijie Shen
          52.
          Reliable TimelineClient Sub-task Resolved Zhijie Shen
          53.
          Cross Origin Filter Http response split vulnerability protection rejects valid origins Sub-task Closed Jonathan Eagles
          54.
          Cross Origin Filter throws UnsupportedOperationException upon destroy Sub-task Closed Mit Desai
          55.
          Tool to measure the performance of the timeline server Sub-task Resolved Chang Li
          56.
          Document of timeline server needs to be updated Sub-task Resolved Unassigned
          57.
          Add retry for timeline client put APIs Sub-task Closed Li Lu
          58.
          Add retry for timeline client getDelegationToken method Sub-task Closed Li Lu
          59.
          RM uses conf instead of token service address to renew timeline delegation tokens Sub-task Closed Jonathan Eagles
          60.
          Add Rolling Time To Lives Level DB Plugin Capabilities Sub-task Resolved Jonathan Eagles
          61.
          Add version info on timeline service / generic history web UI and REST API Sub-task Resolved Zhijie Shen

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                vinodkv Vinod Kumar Vavilapalli
              • Votes:
                0 Vote for this issue
                Watchers:
                75 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: