Details

      Description

      This is an umbrella JIRA for the beta 1 milestone for YARN Timeline Service v.2.

      YARN-2928 was alpha1, YARN-5355 was alpha2.

        Attachments

          Issue Links

          1.
          Limit the flow name size & consider cleanup for hex chars Sub-task Resolved Sushil Ks
          2.
          Ship single node HBase config option with single startup command Sub-task Open Vrushali C
          3.
          Explore supporting a simpler back-end implementation for ATS v2 Sub-task Open Unassigned
          4.
          Bubble exceptions and errors all the way up the calls, including to clients. Sub-task Open Haibo Chen
          5.
          [Collector implementation] Implement timeline collector functionality to serve ATS readers' requests for live apps Sub-task Open Naganarasimha G R
          6.
          [Collector wireup] Implement NM starting the standalone timeline collector daemon Sub-task Open Li Lu
          7.
          [Collector wireup] Handle timeline collector failure scenarios Sub-task Open Varun Saxena
          8.
          [Collector wireup] Work-preserving restarting of per-node timeline collector Sub-task Open Junping Du
          9.
          [Data Serving] Clustering of ATS reader instances Sub-task Open Sunil Govindan
          10.
          [Source organization] Decide detailed package structures for timeline service v2 components Sub-task Open Li Lu
          11.
          [Compatibility] Make TS next gen be compatible with the current TS Sub-task Open Junping Du
          12.
          [Event producers] Change distributed shell to use TIMELINE_SERVICE_VERSION Sub-task Open Li Lu
          13.
          provide RPC metrics via JMX for timeline collectors and readers Sub-task Resolved Vrushali C
          14.
          [Security] users should not be able to create a generic TimelineEntity and associate arbitrary type Sub-task Open Haibo Chen
          15.
          CollectorNodemanagerProtocol's authorization doesn't work Sub-task Resolved Zhijie Shen
          16.
          AM get timeline service info from RM rather than Application specific configuration. Sub-task Patch Available Abhishek Modi
          17.
          Investigate the concurrency issue with the map of timeline collector Sub-task Open Li Lu
          18.
          Timeline entity uniqueness Sub-task Open Vrushali C
          19.
          determine how to generate YARN container events Sub-task Open Naganarasimha G R
          20.
          Enable application client to communicate with new timeline service Sub-task Open Zhijie Shen
          21.
          Consider concurrency situations for TimelineWriter Sub-task Open Unassigned
          22.
          consider nulling member maps/sets of TimelineEntity Sub-task Open Vrushali C
          23.
          [Aggregation] Application/Flow/User/Queue Level Aggregations Sub-task Open Junping Du
          24.
          [Aggregation] Flow and User level aggregation on Application States table Sub-task In Progress Li Lu
          25.
          [Aggregation] Queue-level Aggregation on Application States table Sub-task Open Junping Du
          26.
          Scalability validation of RM writing app/attempt/container lifecycle events Sub-task Open Unassigned
          27.
          Backward compatibility of reader with ATSv1 Sub-task Open Varun Saxena
          28.
          TimelineReader Web UI Implementation Sub-task Resolved Varun Saxena
          29.
          Optimize and synchronize FS Reader and Writer Implementations Sub-task Open Varun Saxena
          30.
          [Fault tolerance] Fault tolerant writer for timeline v2 Sub-task In Progress Joep Rottinghuis
          31.
          Writing more RM side app-level metrics Sub-task Open Naganarasimha G R
          32.
          Writing RM cluster-level metrics Sub-task Open Zhijie Shen
          33.
          Support ACLs in ATSv2 Sub-task In Progress Vrushali C
          34.
          create the flow-version table Sub-task Open Vrushali C
          35.
          offline collector: support timeline clients not associated with an application Sub-task Open Rohith Sharma K S
          36.
          For long running apps (> 2 days), populate flow activity table Sub-task Open Varun Saxena
          37.
          refactor ColumnHelper read* methods Sub-task Open Vrushali C
          38.
          Ensure the final values for metrics/events are emitted/stored at APP completion time Sub-task Open Vrushali C
          39.
          Flow page for Web UI Sub-task Resolved Varun Saxena
          40.
          fix the order of timelinereader in yarn/yarn.cmd Sub-task Open Varun Saxena
          41.
          Need to support Application History Server on ATSV2 Sub-task Open Naganarasimha G R
          42.
          Support Multiple versions of the timeline service at the same time Sub-task Open Unassigned
          43.
          Limit flow runs returned while querying flows Sub-task Open Varun Saxena
          44.
          Retrospect on defaults for created time while querying Sub-task Open Varun Saxena
          45.
          Compaction coprocessor enhancements: On/Off, whitelisting, blacklisting Sub-task Open Vrushali C
          46.
          Issues with HBaseTimelineWriterImpl in single node hadoop & hbase cluster Sub-task Open Vrushali C
          47.
          Enhance hbase coprocessor aggregation operations:GLOBAL_MIN, LATEST_MIN etc and FINAL attributes Sub-task Open Vrushali C
          48.
          Have a separate NM timeline publishing-interval Sub-task Open Naganarasimha G R
          49.
          Support metric filters for flow runs Sub-task In Progress Varun Saxena
          50.
          address HBase compatibility issues with trunk Sub-task Open Sangjin Lee
          51.
          some YARN container events have timestamp of -1 Sub-task Resolved Haibo Chen
          52.
          Review / Recommendations for hbase writer code Sub-task Open Vrushali C
          53.
          Timeline service v2 integration with Federation Sub-task Open Abhishek Modi
          54.
          Consider adding a config setting to accept logical cluster name Sub-task Open Sushil Ks
          55.
          Support long running apps handling multiple flows Sub-task Open Varun Saxena
          56.
          [Atsv2] Support streaming reader API to fetch entities Sub-task Open Rohith Sharma K S
          57.
          Change collector discovery to support collectors mapped to clients but not applications Sub-task Open Li Lu
          58.
          Serve aggregated logs of historical apps from timeline service Sub-task Resolved Rohith Sharma K S
          59.
          Support UNIT for TimelineMetric Sub-task Open Rohith Sharma K S
          60.
          [ATSv2] Create HBase connection only if an app collector is publishing from NM Sub-task Open Varun Saxena
          61.
          Coprocessor related code changes/cleanup pending HBASE-17273 Sub-task Open Vrushali C
          62.
          Support for listing all applications i.e /apps Sub-task Open Rohith Sharma K S
          63.
          Inline java doc with Hadoop formatter Sub-task Open WEI-HSIAO-LEE
          64.
          Support substring match in filters Sub-task Open Prabhu Joseph
          65.
          create a REST API that returns the clusters for a given app id Sub-task Open Sushil Ks
          66.
          Support for new REST end point /clusterids Sub-task Open Sushil Ks
          67.
          [Storage implementation] Create HDFS backing storage implementation for ATS reads Sub-task Resolved Abhishek Modi
          68.
          Launch app collectors for unmanaged AMs' Sub-task Open Abhishek Modi
          69.
          Ensure timeline service v2 codebase gets UGI from HttpServletRequest in a consistent way Sub-task Resolved Abhishek Modi
          70.
          [ATSv2] : NPE for recovered finished applications in NM during upgrade Sub-task Resolved Unassigned
          71.
          Make the aggregation interval in AppLevelTimelineCollector configurable Sub-task Resolved Abhishek Modi
          72.
          [ATSv2] Fix findbugs warnings Sub-task Resolved Abhishek Modi
          73.
          Timeline client won't be able to write when TimelineCollector is not up yet, or NM is down Sub-task Open Vrushali C
          74.
          Consider writing to both ats v1 & v2 from RM for smoother upgrades Sub-task Resolved Aaron Gresch
          75.
          Have a way to turn off container metrics from NMs Sub-task Resolved Abhishek Modi
          76.
          Address race condition on TimelineWriter.flush() caused by buffer-sized flush Sub-task Open Haibo Chen
          77.
          Support for listing flows with filter userid Sub-task Patch Available Haibo Chen
          78.
          Support for collapse filter for flow entities with in the boundary of date range. Sub-task Open Unassigned
          79.
          Use app user to publish entities from NM Sub-task Patch Available Varun Saxena
          80.
          [Storage implementation] Adding retry semantics to HDFS backing storage Sub-task Resolved Abhishek Modi
          81.
          Allow port range to be specified while starting NM Timeline collector manager. Sub-task Resolved Abhishek Modi
          82.
          start time key in NM leveldb store should be removed when container is removed Sub-task Resolved Ajith S
          83.
          [ATSv2 Security] Delegation token support for Timeline reader Sub-task Open Varun Saxena
          84.
          [ATSv2 Security] Consider using delegation token for publishing entities from NM Sub-task Open Abhishek Modi
          85.
          [ATSv2 Security] Use token for communication between NM and node collector Sub-task Open Unassigned
          86.
          Documentation updates for branch2 Timeline Service V2 Sub-task Open Unassigned
          87.
          Extend aggregation operation for new ATS design Sub-task Open Linlin Zhou
          88.
          Fix findbugs warning in timeline service on branch-2 Sub-task Resolved Vrushali C
          89.
          The error message in TimelineSchemaCreator is not enough to find out the error. Sub-task Resolved Jinjiang Ling
          90.
          Backport new yarn-ui to branch2 code (starting with YARN-5355_branch2) Sub-task Resolved Vrushali C
          91.
          TestTimelineReaderWebServicesHBaseStorage fails with NoClassDefFoundError on TSv2 branch2 Sub-task Resolved Varun Saxena
          92.
          Ensure only NM classpath in 2.x gets TSv2 related hbase jars, not the user classpath Sub-task Resolved Varun Saxena
          93.
          Add a yarn application cost calculation framework in TimelineService v2 Sub-task Open Sushil Ks
          94.
          Enable timeline collector fault tolerance Sub-task Open Rohith Sharma K S
          95.
          Backport ATSv2 to branch-2 Sub-task Resolved Varun Saxena
          96.
          [atsv2 read acls] Include domain table creation as part of schema creator Sub-task Resolved Vrushali C
          97.
          TestTimelineReaderWebServices times out on YARN-5355_branch2 Sub-task Open Unassigned
          98.
          Documentation changes post branch-2 merge Sub-task Resolved Vrushali C
          99.
          Update LICENSE.txt with respect to new YARN UI merge Sub-task Resolved Varun Saxena
          100.
          [atsv2 read acls] REST API to list domain/domains Sub-task Patch Available Abhishek Modi
          101.
          [Atsv2] load hbase configuration from filesystem rather than URL Sub-task Resolved Prabhu Joseph
          102.
          YARN_TIMELINESERVER_OPTS is not valid env variable for TimelineReaderServer Sub-task Resolved Haibo Chen
          103.
          metricsTimeStart and metricsTimeEnd should be all lower case in the doc Sub-task Resolved Haibo Chen
          104.
          [Atsv2] Update document for running v1 and v2 TS Sub-task Resolved Suma Shivaprasad
          105.
          FlowRunReader constructs min_start_time filter for both createdtimestart and createdtimeend. Sub-task Resolved Haibo Chen
          106.
          [atsv2 read acls] Add a API TimelineV2Client#putDomain to publish domain information. Sub-task Open Rohith Sharma K S
          107.
          [atsv2 read acls] Add TimelineWriter#writeDomain Sub-task Resolved Rohith Sharma K S
          108.
          Do ACLs check for flow activity entities Sub-task Open Abhishek Modi
          109.
          Do ACLs check for flowrun entities Sub-task Open Abhishek Modi
          110.
          Do ACLs check for application entities Sub-task Open Abhishek Modi
          111.
          Do ACLs check for generic entities Sub-task Open Haibo Chen
          112.
          Do ACLs check for sub app entities Sub-task Open Abhishek Modi
          113.
          Do ACLs check while retrieving entity-types per application Sub-task Patch Available Prabhu Joseph
          114.
          [Atsv2] Retrospect storing entities into sub application table from client perspective Sub-task Resolved Rohith Sharma K S
          115.
          Add table for storing timeline domain information Sub-task Resolved Unassigned
          116.
          Support data retention policy in YARN ATSv2 Sub-task Open Unassigned
          117.
          Improve debugging on ATSv2 reader server Sub-task Open Haibo Chen
          118.
          ATSv2 REST API queries do not return results for uppercase application tags Sub-task Resolved Charan Hebri
          119.
          Allow YARN ATSv2 ACLs to be disabled Sub-task Open Unassigned
          120.
          [atsv2 read acls] REST API to get domain info by id Sub-task Resolved Unassigned
          121.
          Recover domain from backend for TimelineCollector. Sub-task In Progress Haibo Chen
          122.
          YarnClient should contact TimelineReader for application/attempt/container report Sub-task Resolved Abhishek Modi
          123.
          Support HBase 2.0.0-beta1 as ATSv2 backend Sub-task Resolved Unassigned
          124.
          Update timeline collector to store and add domain info to entities while writing Sub-task Open Abhishek Modi
          125.
          Improve error message for invalid value in fields attribute Sub-task Resolved Abhishek Modi
          126.
          Give an informative message when incorrect format is used in ATSv2 filter attributes Sub-task Resolved Rohith Sharma K S
          127.
          Final Status of applications shown as UNDEFINED in ATS app queries Sub-task Reopened Prabhu Joseph
          128.
          ATS v2 returns invalid YARN_CONTAINER_ALLOCATED_HOST_HTTP_ADDRESS from NM Sub-task Resolved Rohith Sharma K S
          129.
          Incorrect HTTP status code returned by ATSv2 for non-whitelisted users Sub-task Resolved Rohith Sharma K S
          130.
          [atsv2 read acls] Update timeline collector to include domain for entities Sub-task Open Abhishek Modi
          131.
          HTTPS Ats v2 api call fails with "bad HTTP parsed" Sub-task Resolved Charan Hebri
          132.
          Race condition when container events are published for KILLED applications Sub-task Resolved Rohith Sharma K S
          133.
          Provide generic log servlet for serving logs Sub-task Open Abhishek Modi
          134.
          Adding JMX Metrics for Timeline Collector and Reader Sub-task Resolved Sushil Ks
          135.
          [atsv2 read acls] Enable hbase writer to store domain/ACLs info Sub-task Open Prabhu Joseph
          136.
          [atsv2 read acls] Update NM to include ACLs for system entities Sub-task Open Prabhu Joseph
          137.
          [atsv2 read acls] Coprocessor for reader authorization check Sub-task In Progress Vrushali C
          138.
          timeline flow runs API createdtimestart/createdtimeend parameter does not work Sub-task Open Unassigned
          139.
          Use hbase shaded jars Sub-task In Progress Vrushali C
          140.
          AMs are unable to publish events if NMCollectorService starts at dynamic port. Sub-task Open Abhishek Modi
          141.
          Provide Java client for fetching Yarn specific entities from TimelineReader Sub-task Resolved Abhishek Modi
          142.
          Document behavior of YARN-5742 Sub-task Resolved Suma Shivaprasad
          143.
          ApplicationCLI should have option to take clusterId Sub-task Resolved Rohith Sharma K S
          144.
          [atsv2] Restrict the number of elements held in timeline collector when backend is unreachable for async calls Sub-task Resolved Abhishek Modi
          145.
          HBaseTimelineWriterImpl sync writes has to avoid thread blocking if storage down Sub-task Resolved Prabhu Joseph
          146.
          Publish container killed, paused and resumed events to ATSv2. Sub-task Resolved Abhishek Modi
          147.
          Publish federation events to ATSv2. Sub-task Open Abhishek Modi
          148.
          ATS v2 Generic TimelineStorageMonitor Sub-task Resolved Prabhu Joseph
          149.
          Username splits won't help timelineservice.app_flow table Sub-task Resolved Prabhu Joseph
          150.
          HBaseTimelineSchemaCreator has to allow user to configure pre-splits Sub-task Patch Available Prabhu Joseph
          151.
          Update document for ATS HBase Custom tablenames (-entityTableName) Sub-task Resolved Prabhu Joseph
          152.
          HBaseTimelineSchemaCreator skipExistingTable should honor custom hbase tablenames Sub-task Resolved Prabhu Joseph
          153.
          FlowActivity and FlowRun table prefix is wrong Sub-task Resolved Prabhu Joseph
          154.
          GET /apps/{appid}/entities/YARN_APPLICATION accesses application table instead of entity table Sub-task Patch Available Prabhu Joseph
          155.
          ATSV2 /apps/appId/entities/YARN_CONTAINER rest api does not show metrics Sub-task Resolved Prabhu Joseph
          156.
          @Path("/apps/{appid}/appattempts") error message misleads Sub-task Patch Available Prabhu Joseph
          157.
          RM Crashes with NPE at TimelineServiceV2Publisher.putEntity Sub-task Resolved Prabhu Joseph
          158.
          Support to alter table properties in HBaseTimelineSchemaCreator Sub-task Open Tarun Parimi
          159.
          Make async/sync writes to timeline service configurable Sub-task Open Unassigned

            Activity

              People

              • Assignee:
                vrushalic Vrushali C
                Reporter:
                vrushalic Vrushali C
              • Votes:
                0 Vote for this issue
                Watchers:
                16 Start watching this issue

                Dates

                • Created:
                  Updated: