Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-9802

YARN Timeline Service v2 (post GA features)

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • None

    Description

      Epic jira to track new feature enhancements or issues after the next release of Timeline Service v2

      Attachments

        Issue Links

          Parent Feature

          New Feature - A new feature of the product, which has yet to be developed. MAPREDUCE-6943 Mapreduce tasks for YARN Timeline Service v.2: (post GA features)

          • Major - Major loss of function.
          • Open
          1.
          		 Support ACLs in ATSv2 Sub-task In Progress Vrushali C  
          2.
          		 Support metric filters for flow runs Sub-task In Progress Varun Saxena  
          3.
          		 Recover domain from backend for TimelineCollector. Sub-task In Progress Haibo Chen  
          4.
          		 [atsv2 read acls] Coprocessor for reader authorization check Sub-task In Progress Vrushali C  
          5.
          		 HBaseTimelineSchemaCreator has to allow user to configure pre-splits Sub-task Patch Available Ashutosh Gupta  
          6.
          		 AM get timeline service info from RM rather than Application specific configuration. Sub-task Patch Available Abhishek Modi  
          7.
          		 Use hbase shaded jars Sub-task In Progress Vrushali C  
          8.
          		 Use app user to publish entities from NM Sub-task Patch Available Varun Saxena  
          9.
          		 Support for listing flows with filter userid Sub-task Patch Available Haibo Chen  
          10.
          		 [Fault tolerance] Fault tolerant writer for timeline v2 Sub-task In Progress Joep Rottinghuis  
          11.
          		 [atsv2 read acls] REST API to list domain/domains Sub-task Patch Available Abhishek Modi  
          12.
          		 @Path("/apps/{appid}/appattempts") error message misleads Sub-task Patch Available Ashutosh Gupta  
          13.
          		 Do ACLs check for flowrun entities Sub-task Open Abhishek Modi  
          14.
          		 Enable timeline collector fault tolerance Sub-task Open Unassigned  
          15.
          		 [atsv2 read acls] Update timeline collector to include domain for entities Sub-task Open Abhishek Modi  
          16.
          		 [atsv2 read acls] Update NM to include ACLs for system entities Sub-task Open Prabhu Joseph  
          17.
          		 [Security] users should not be able to create a generic TimelineEntity and associate arbitrary type Sub-task Open Haibo Chen  
          18.
          		 Do ACLs check for flow activity entities Sub-task Open Abhishek Modi  
          19.
          		 [ATSv2 Security] Delegation token support for Timeline reader Sub-task Open Varun Saxena  
          20.
          		 Do ACLs check for generic entities Sub-task Open Haibo Chen  
          21.
          		 Support for new REST end point /clusterids Sub-task Open Sushil Ks  
          22.
          		 [atsv2 read acls] Enable hbase writer to store domain/ACLs info Sub-task Open Prabhu Joseph  
          23.
          		 [atsv2 read acls] Add a API TimelineV2Client#putDomain to publish domain information. Sub-task Open Unassigned  
          24.
          		 Allow YARN ATSv2 ACLs to be disabled Sub-task Open Unassigned  
          25.
          		 Timeline service v2 integration with Federation Sub-task Open Abhishek Modi  
          26.
          		 Add a yarn application cost calculation framework in TimelineService v2 Sub-task Open Sushil Ks  
          27.
          		 Do ACLs check for sub app entities Sub-task Open Abhishek Modi  
          28.
          		 [ATSv2 Security] Use token for communication between NM and node collector Sub-task Open Unassigned  
          29.
          		 [ATSv2 Security] Consider using delegation token for publishing entities from NM Sub-task Open Abhishek Modi  
          30.
          		 create a REST API that returns the clusters for a given app id Sub-task Open Sushil Ks  
          31.
          		 Do ACLs check for application entities Sub-task Open Abhishek Modi  
          32.
          		 Support substring match in filters Sub-task Open Prabhu Joseph  
          33.
          		 Publish federation events to ATSv2. Sub-task Open Minni Mittal  
          34.
          		 Launch app collectors for unmanaged AMs' Sub-task Open Abhishek Modi  
          35.
          		 Coprocessor related code changes/cleanup pending HBASE-17273 Sub-task Open Vrushali C  
          36.
          		 Improve debugging on ATSv2 reader server Sub-task Open Haibo Chen  
          37.
          		 Final Status of applications shown as UNDEFINED in ATS app queries Sub-task Reopened Prabhu Joseph  
          38.
          		 Support data retention policy in YARN ATSv2 Sub-task Open Unassigned  
          39.
          		 Writing more RM side app-level metrics Sub-task Open Naganarasimha G R  
          40.
          		 Support for collapse filter for flow entities with in the boundary of date range. Sub-task Open Unassigned  
          41.
          		 Review / Recommendations for hbase writer code Sub-task Open Vrushali C  
          42.
          		 Issues with HBaseTimelineWriterImpl in single node hadoop & hbase cluster Sub-task Open Vrushali C  
          43.
          		 refactor ColumnHelper read* methods Sub-task Open Vrushali C  
          44.
          		 Support for listing all applications i.e /apps Sub-task Open Unassigned  
          45.
          		 Documentation updates for branch2 Timeline Service V2 Sub-task Open Unassigned  
          46.
          		 Timeline entity uniqueness Sub-task Open Vrushali C  
          47.
          		 Limit flow runs returned while querying flows Sub-task Open Varun Saxena  
          48.
          		 Support Multiple versions of the timeline service at the same time Sub-task Open Unassigned  
          49.
          		 Update timeline collector to store and add domain info to entities while writing Sub-task Open Abhishek Modi  
          50.
          		 Scalability validation of RM writing app/attempt/container lifecycle events Sub-task Open Unassigned  
          51.
          		 AMs are unable to publish events if NMCollectorService starts at dynamic port. Sub-task Open Abhishek Modi  
          52.
          		 Compaction coprocessor enhancements: On/Off, whitelisting, blacklisting Sub-task Open Vrushali C  
          53.
          		 Address race condition on TimelineWriter.flush() caused by buffer-sized flush Sub-task Open Yousef Abu-Salah  
          54.
          		 Bubble exceptions and errors all the way up the calls, including to clients. Sub-task Open Haibo Chen  
          55.
          		 Extend aggregation operation for new ATS design Sub-task Open Linlin Zhou  
          56.
          		 timeline flow runs API createdtimestart/createdtimeend parameter does not work Sub-task Open Unassigned  
          57.
          		 Writing RM cluster-level metrics Sub-task Open Zhijie Shen  
          58.
          		 Support UNIT for TimelineMetric Sub-task Open Unassigned  
          59.
          		 Provide generic log servlet for serving logs Sub-task Open Abhishek Modi  
          60.
          		 Make async/sync writes to timeline service configurable Sub-task Open Unassigned  
          61.
          		 create the flow-version table Sub-task Open Vrushali C  
          62.
          		 Consider adding a config setting to accept logical cluster name Sub-task Open Sushil Ks  
          63.
          		 Optimize and synchronize FS Reader and Writer Implementations Sub-task Open Varun Saxena  
          64.
          		 Timeline client won't be able to write when TimelineCollector is not up yet, or NM is down Sub-task Open Vrushali C  
          65.
          		 Inline java doc with Hadoop formatter Sub-task Open WEI-HSIAO-LEE  
          66.
          		 Ensure the final values for metrics/events are emitted/stored at APP completion time Sub-task Open Vrushali C  
          67.
          		 Support to alter table properties in HBaseTimelineSchemaCreator Sub-task Open Tarun Parimi  
          68.
          		 Enhance hbase coprocessor aggregation operations:GLOBAL_MIN, LATEST_MIN etc and FINAL attributes Sub-task Open Vrushali C  
          69.
          		 TestTimelineReaderWebServices times out on YARN-5355_branch2 Sub-task Open Unassigned  
          70.
          		 Have a separate NM timeline publishing-interval Sub-task Open Naganarasimha G R  
          71.
          		 Change collector discovery to support collectors mapped to clients but not applications Sub-task Open Li Lu  
          72.
          		 offline collector: support timeline clients not associated with an application Sub-task Open Unassigned  
          73.
          		 [Atsv2] Support streaming reader API to fetch entities Sub-task Open Unassigned  
          74.
          		 Support long running apps handling multiple flows Sub-task Open Varun Saxena  
          75.
          		 TimelineService Hbase tests are failing Sub-task Open Prabhu Joseph  
          76.
          		 TimelineSchemaCreator breaks after protobuffer upgrade Sub-task Open Unassigned  
          77.
          		 Region in ats-hbase table 'prod.timelineservice.entity' failing to split Sub-task Open Unassigned  
          78.
          		 FlowRunCoprocessor Scan Used Deprecated Method Sub-task In Progress Shilun Fan

          100%
          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 50m

          Activity

            People

              Unassigned Unassigned
              vrushalic Vrushali C
              Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 2h 10m
                  2h 10m