Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-9802

YARN Timeline Service v2 (post GA features)

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • None

    Description

      Epic jira to track new feature enhancements or issues after the next release of Timeline Service v2

      Attachments

        Issue Links

          1.
          Support ACLs in ATSv2 Sub-task In Progress Vrushali C  
          2.
          Support metric filters for flow runs Sub-task In Progress Varun Saxena  
          3.
          RM crashes due to runtime errors in TimelineServiceV2Publisher Sub-task Resolved Tarun Parimi  
          4.
          Recover domain from backend for TimelineCollector. Sub-task In Progress Haibo Chen  
          5.
          [atsv2 read acls] Coprocessor for reader authorization check Sub-task In Progress Vrushali C  
          6.
          HBaseTimelineSchemaCreator has to allow user to configure pre-splits Sub-task Patch Available Ashutosh Gupta  
          7.
          AM get timeline service info from RM rather than Application specific configuration. Sub-task Patch Available Abhishek Modi  
          8.
          GET /apps/{appid}/entities/YARN_APPLICATION accesses application table instead of entity table Sub-task Resolved Ashutosh Gupta

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 40m
          9.
          Use hbase shaded jars Sub-task In Progress Vrushali C  
          10.
          Use app user to publish entities from NM Sub-task Patch Available Varun Saxena  
          11.
          Support for listing flows with filter userid Sub-task Patch Available Haibo Chen  
          12.
          [Fault tolerance] Fault tolerant writer for timeline v2 Sub-task In Progress Joep Rottinghuis  
          13.
          Do ACLs check while retrieving entity-types per application Sub-task Resolved Prabhu Joseph  
          14.
          [atsv2 read acls] REST API to list domain/domains Sub-task Patch Available Abhishek Modi  
          15.
          @Path("/apps/{appid}/appattempts") error message misleads Sub-task Patch Available Ashutosh Gupta  
          16.
          Do ACLs check for flowrun entities Sub-task Open Abhishek Modi  
          17.
          Enable timeline collector fault tolerance Sub-task Open Unassigned  
          18.
          [atsv2 read acls] Update timeline collector to include domain for entities Sub-task Open Abhishek Modi  
          19.
          [atsv2 read acls] Update NM to include ACLs for system entities Sub-task Open Prabhu Joseph  
          20.
          [Security] users should not be able to create a generic TimelineEntity and associate arbitrary type Sub-task Open Haibo Chen  
          21.
          Do ACLs check for flow activity entities Sub-task Open Abhishek Modi  
          22.
          [ATSv2 Security] Delegation token support for Timeline reader Sub-task Open Varun Saxena  
          23.
          Do ACLs check for generic entities Sub-task Open Haibo Chen  
          24.
          Support for new REST end point /clusterids Sub-task Open Sushil Ks  
          25.
          [atsv2 read acls] Enable hbase writer to store domain/ACLs info Sub-task Open Prabhu Joseph  
          26.
          [atsv2 read acls] Add a API TimelineV2Client#putDomain to publish domain information. Sub-task Open Unassigned  
          27.
          Allow YARN ATSv2 ACLs to be disabled Sub-task Open Unassigned  
          28.
          Timeline service v2 integration with Federation Sub-task Open Abhishek Modi  
          29.
          Add a yarn application cost calculation framework in TimelineService v2 Sub-task Open Sushil Ks  
          30.
          Do ACLs check for sub app entities Sub-task Open Abhishek Modi  
          31.
          [ATSv2 Security] Use token for communication between NM and node collector Sub-task Open Unassigned  
          32.
          [ATSv2 Security] Consider using delegation token for publishing entities from NM Sub-task Open Abhishek Modi  
          33.
          create a REST API that returns the clusters for a given app id Sub-task Open Sushil Ks  
          34.
          Do ACLs check for application entities Sub-task Open Abhishek Modi  
          35.
          Support substring match in filters Sub-task Open Prabhu Joseph  
          36.
          Publish federation events to ATSv2. Sub-task Open Minni Mittal  
          37.
          Launch app collectors for unmanaged AMs' Sub-task Open Abhishek Modi  
          38.
          Coprocessor related code changes/cleanup pending HBASE-17273 Sub-task Open Vrushali C  
          39.
          Improve debugging on ATSv2 reader server Sub-task Open Haibo Chen  
          40.
          Final Status of applications shown as UNDEFINED in ATS app queries Sub-task Reopened Prabhu Joseph  
          41.
          Support data retention policy in YARN ATSv2 Sub-task Open Unassigned  
          42.
          Writing more RM side app-level metrics Sub-task Open Naganarasimha G R  
          43.
          Support for collapse filter for flow entities with in the boundary of date range. Sub-task Open Unassigned  
          44.
          Review / Recommendations for hbase writer code Sub-task Open Vrushali C  
          45.
          Issues with HBaseTimelineWriterImpl in single node hadoop & hbase cluster Sub-task Open Vrushali C  
          46.
          refactor ColumnHelper read* methods Sub-task Open Vrushali C  
          47.
          fix the order of timelinereader in yarn/yarn.cmd Sub-task Resolved Varun Saxena  
          48.
          Support for listing all applications i.e /apps Sub-task Open Unassigned  
          49.
          Documentation updates for branch2 Timeline Service V2 Sub-task Open Unassigned  
          50.
          address HBase compatibility issues with trunk Sub-task Resolved Sangjin Lee  
          51.
          Timeline entity uniqueness Sub-task Open Vrushali C  
          52.
          Limit flow runs returned while querying flows Sub-task Open Varun Saxena  
          53.
          Support Multiple versions of the timeline service at the same time Sub-task Open Unassigned  
          54.
          Need to support Application History Server on ATSV2 Sub-task Resolved Naganarasimha G R  
          55.
          Update timeline collector to store and add domain info to entities while writing Sub-task Open Abhishek Modi  
          56.
          Scalability validation of RM writing app/attempt/container lifecycle events Sub-task Open Unassigned  
          57.
          Backward compatibility of reader with ATSv1 Sub-task Resolved Varun Saxena  
          58.
          AMs are unable to publish events if NMCollectorService starts at dynamic port. Sub-task Open Abhishek Modi  
          59.
          Compaction coprocessor enhancements: On/Off, whitelisting, blacklisting Sub-task Open Vrushali C  
          60.
          Address race condition on TimelineWriter.flush() caused by buffer-sized flush Sub-task Open Yousef Abu-Salah  
          61.
          Bubble exceptions and errors all the way up the calls, including to clients. Sub-task Open Haibo Chen  
          62.
          Extend aggregation operation for new ATS design Sub-task Open Linlin Zhou  
          63.
          timeline flow runs API createdtimestart/createdtimeend parameter does not work Sub-task Open Unassigned  
          64.
          For long running apps (> 2 days), populate flow activity table Sub-task Resolved Varun Saxena  
          65.
          Writing RM cluster-level metrics Sub-task Open Zhijie Shen  
          66.
          Support UNIT for TimelineMetric Sub-task Open Unassigned  
          67.
          Provide generic log servlet for serving logs Sub-task Open Abhishek Modi  
          68.
          Make async/sync writes to timeline service configurable Sub-task Open Unassigned  
          69.
          create the flow-version table Sub-task Open Vrushali C  
          70.
          Consider adding a config setting to accept logical cluster name Sub-task Open Sushil Ks  
          71.
          Optimize and synchronize FS Reader and Writer Implementations Sub-task Open Varun Saxena  
          72.
          Timeline client won't be able to write when TimelineCollector is not up yet, or NM is down Sub-task Open Vrushali C  
          73.
          Inline java doc with Hadoop formatter Sub-task Open WEI-HSIAO-LEE  
          74.
          Ensure the final values for metrics/events are emitted/stored at APP completion time Sub-task Open Vrushali C  
          75.
          Support to alter table properties in HBaseTimelineSchemaCreator Sub-task Open Tarun Parimi  
          76.
          Enhance hbase coprocessor aggregation operations:GLOBAL_MIN, LATEST_MIN etc and FINAL attributes Sub-task Open Vrushali C  
          77.
          TestTimelineReaderWebServices times out on YARN-5355_branch2 Sub-task Open Unassigned  
          78.
          Have a separate NM timeline publishing-interval Sub-task Open Naganarasimha G R  
          79.
          [ATSv2] Create HBase connection only if an app collector is publishing from NM Sub-task Resolved Varun Saxena  
          80.
          Change collector discovery to support collectors mapped to clients but not applications Sub-task Open Li Lu  
          81.
          offline collector: support timeline clients not associated with an application Sub-task Open Unassigned  
          82.
          Retrospect on defaults for created time while querying Sub-task Resolved Varun Saxena  
          83.
          [Atsv2] Support streaming reader API to fetch entities Sub-task Open Unassigned  
          84.
          Support long running apps handling multiple flows Sub-task Open Varun Saxena  
          85.
          NM hangs at serviceStop when ATSV2 Backend Hbase is Down Sub-task Resolved Prabhu Joseph  
          86.
          TimelineCollectorWebService#putEntities blocked when ATSV2 HBase is down. Sub-task Resolved Ashutosh Gupta

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 40m
          87.
          TimelineService Hbase tests are failing Sub-task Open Prabhu Joseph  
          88.
          TimelineSchemaCreator breaks after protobuffer upgrade Sub-task Open Unassigned  
          89.
          TimelineV2Client may leak file descriptors creating ClientResponse objects. Sub-task Resolved Anand Srinivasan  
          90.
          Region in ats-hbase table 'prod.timelineservice.entity' failing to split Sub-task Open Unassigned  
          91.
          TimelineConnector does not destroy the jersey client Sub-task Resolved Tanu Ajmera  
          92.
          Prevent Fatal CancelledException in TimelineV2Client when stopping Sub-task Resolved Tarun Parimi  
          93.
          TestRMHATimelineCollectors fails on hadoop trunk Sub-task Resolved Bilwa S T  
          94.
          FlowRunCoprocessor Scan Used Deprecated Method Sub-task Open Unassigned

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 50m

          Activity

            People

              Unassigned Unassigned
              vrushalic Vrushali C
              Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 2h 10m
                  2h 10m