Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4233

YARN Timeline Service plugin: ATS v1.5

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: timelineserver
    • Labels:
      None

      Description

      Copy the description from YARN-3942: Timeline store to read events from HDFS
      This adds a new timeline store plugin that is intended as a stop-gap measure to mitigate some of the issues we've seen with ATS v1 while waiting for ATS v2. The intent of this plugin is to provide a workable solution for running the Tez UI against the timeline server on a large-scale clusters running many thousands of jobs per day.

        Attachments

          Issue Links

          1.
          Timeline store to read events from HDFS Sub-task Resolved Jason Darrell Lowe
          2.
          New levelDB cache storage for timeline v1.5 Sub-task Resolved Li Lu
          3.
          New put APIs in TimelineClient for ats v1.5 Sub-task Resolved Xuan Gong
          4.
          Provide new timeline plugin storage to support fine-grained entity caching Sub-task Resolved Li Lu
          5.
          Document new put APIs in TimelineClient for ATS 1.5 Sub-task Resolved Xuan Gong
          6.
          Allow YARN distributed shell to use ATS v1.5 APIs Sub-task Resolved Li Lu
          7.
          TimerTasks leak in ATS V1.5 Writer Sub-task Resolved Xuan Gong
          8.
          Allow YARN system metrics publisher to use ATS v1.5 APIs Sub-task Open Li Lu
          9.
          Document ATS v1.5 Sub-task Open Li Lu
          10.
          EntityGroupFSTimelineStore to not log errors during shutdown Sub-task Resolved Li Lu
          11.
          Improving EntityGroupFSTimelineStore on exception handling, test setup, and concurrency Sub-task Resolved Steve Loughran
          12.
          ATS 1.5 parse pipeline to consider handling open() events recoverably Sub-task Open Unassigned
          13.
          TimelineClient to implement Flushable; propagate to writer Sub-task Resolved Unassigned
          14.
          ATS 1.5 timelineclient impl call flush after every event write Sub-task Resolved Xuan Gong
          15.
          ATS 1.5 timelineclient impl try to create attempt directory for every event call Sub-task Resolved Xuan Gong
          16.
          Change Log Level to DEBUG for putDomain call in ATS 1.5 Sub-task Resolved Xuan Gong
          17.
          Metric improvements for ATS v1.5 storage components Sub-task Resolved Li Lu
          18.
          Add HDFS caller context for EntityGroupFSTimelineStore Sub-task Resolved Li Lu
          19.
          EntityGroupFSTimelineStore to close cached leveldb instances in serviceStop() Sub-task Resolved Unassigned
          20.
          findbug issue on LevelDBCacheTimelineStore Sub-task Resolved Zhiyuan Yang
          21.
          Read cache concurrency issue between read and evict in EntityGroupFS timeline store Sub-task Resolved Li Lu
          22.
          Support for specifying a path for ATS plugin jars Sub-task Resolved Li Lu
          23.
          Support for specifying a path for ATS plugin jars Sub-task Resolved Li Lu
          24.
          EntityGroupFSTimelineStore should allow configurable cache stores. Sub-task Resolved Li Lu

            Activity

              People

              • Assignee:
                xgong Xuan Gong
                Reporter:
                xgong Xuan Gong
              • Votes:
                0 Vote for this issue
                Watchers:
                16 Start watching this issue

                Dates

                • Created:
                  Updated: