Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-7055

YARN Timeline Service v.2: beta 1 / GA

    XMLWordPrintableJSON

    Details

    • Release Note:
      Application Timeline Server v2 is ready for production. It is GA from 3.2.1 release on wards.

      Description

      This is an umbrella JIRA for the beta 1 milestone for YARN Timeline Service v.2.

      YARN-2928 was alpha1, YARN-5355 was alpha2.

        Attachments

        1. TSv2 next steps.pdf
          81 kB
          Vrushali C

          Issue Links

          1.
          Limit the flow name size & consider cleanup for hex chars Sub-task Resolved Sushil Ks
          2.
          Ship single node HBase config option with single startup command Sub-task Resolved Vrushali C
          3.
          Explore supporting a simpler back-end implementation for ATS v2 Sub-task Resolved Unassigned
          4.
          [Collector implementation] Implement timeline collector functionality to serve ATS readers' requests for live apps Sub-task Resolved Naganarasimha G R
          5.
          [Collector wireup] Implement NM starting the standalone timeline collector daemon Sub-task Resolved Li Lu
          6.
          [Collector wireup] Handle timeline collector failure scenarios Sub-task Resolved Varun Saxena
          7.
          [Collector wireup] Work-preserving restarting of per-node timeline collector Sub-task Resolved Junping Du
          8.
          [Data Serving] Clustering of ATS reader instances Sub-task Resolved Sunil G
          9.
          [Source organization] Decide detailed package structures for timeline service v2 components Sub-task Resolved Li Lu
          10.
          [Compatibility] Make TS next gen be compatible with the current TS Sub-task Resolved Junping Du
          11.
          [Event producers] Change distributed shell to use TIMELINE_SERVICE_VERSION Sub-task Resolved Li Lu
          12.
          provide RPC metrics via JMX for timeline collectors and readers Sub-task Resolved Vrushali C
          13.
          CollectorNodemanagerProtocol's authorization doesn't work Sub-task Resolved Zhijie Shen
          14.
          Investigate the concurrency issue with the map of timeline collector Sub-task Resolved Li Lu
          15.
          determine how to generate YARN container events Sub-task Resolved Naganarasimha G R
          16.
          Enable application client to communicate with new timeline service Sub-task Resolved Zhijie Shen
          17.
          Consider concurrency situations for TimelineWriter Sub-task Resolved Unassigned
          18.
          consider nulling member maps/sets of TimelineEntity Sub-task Resolved Vrushali C
          19.
          [Aggregation] Application/Flow/User/Queue Level Aggregations Sub-task Resolved Junping Du
          20.
          [Aggregation] Flow and User level aggregation on Application States table Sub-task Resolved Li Lu
          21.
          [Aggregation] Queue-level Aggregation on Application States table Sub-task Resolved Junping Du
          22.
          TimelineReader Web UI Implementation Sub-task Resolved Varun Saxena
          23.
          Flow page for Web UI Sub-task Resolved Varun Saxena
          24.
          some YARN container events have timestamp of -1 Sub-task Resolved Haibo Chen
          25.
          Serve aggregated logs of historical apps from timeline service Sub-task Resolved Rohith Sharma K S
          26.
          [Storage implementation] Create HDFS backing storage implementation for ATS reads Sub-task Resolved Abhishek Modi
          27.
          Ensure timeline service v2 codebase gets UGI from HttpServletRequest in a consistent way Sub-task Resolved Abhishek Modi
          28.
          [ATSv2] : NPE for recovered finished applications in NM during upgrade Sub-task Resolved Unassigned
          29.
          Make the aggregation interval in AppLevelTimelineCollector configurable Sub-task Resolved Abhishek Modi
          30.
          [ATSv2] Fix findbugs warnings Sub-task Resolved Abhishek Modi
          31.
          Consider writing to both ats v1 & v2 from RM for smoother upgrades Sub-task Resolved Aaron Gresch
          32.
          Have a way to turn off container metrics from NMs Sub-task Resolved Abhishek Modi
          33.
          Update ATSv2 document for latest feature supports Sub-task Resolved Rohith Sharma K S
          34.
          [Storage implementation] Adding retry semantics to HDFS backing storage Sub-task Resolved Abhishek Modi
          35.
          Allow port range to be specified while starting NM Timeline collector manager. Sub-task Resolved Abhishek Modi
          36.
          start time key in NM leveldb store should be removed when container is removed Sub-task Resolved Ajith S
          37.
          Fix findbugs warning in timeline service on branch-2 Sub-task Resolved Vrushali C
          38.
          The error message in TimelineSchemaCreator is not enough to find out the error. Sub-task Resolved Jinjiang Ling
          39.
          Backport new yarn-ui to branch2 code (starting with YARN-5355_branch2) Sub-task Resolved Vrushali C
          40.
          TestTimelineReaderWebServicesHBaseStorage fails with NoClassDefFoundError on TSv2 branch2 Sub-task Resolved Varun Saxena
          41.
          Ensure only NM classpath in 2.x gets TSv2 related hbase jars, not the user classpath Sub-task Resolved Varun Saxena
          42.
          Backport ATSv2 to branch-2 Sub-task Resolved Varun Saxena
          43.
          [atsv2 read acls] Include domain table creation as part of schema creator Sub-task Resolved Vrushali C
          44.
          Documentation changes post branch-2 merge Sub-task Resolved Vrushali C
          45.
          Update LICENSE.txt with respect to new YARN UI merge Sub-task Resolved Varun Saxena
          46.
          [Atsv2] load hbase configuration from filesystem rather than URL Sub-task Resolved Prabhu Joseph
          47.
          YARN_TIMELINESERVER_OPTS is not valid env variable for TimelineReaderServer Sub-task Resolved Haibo Chen
          48.
          metricsTimeStart and metricsTimeEnd should be all lower case in the doc Sub-task Resolved Haibo Chen
          49.
          [Atsv2] Update document for running v1 and v2 TS Sub-task Resolved Suma Shivaprasad
          50.
          FlowRunReader constructs min_start_time filter for both createdtimestart and createdtimeend. Sub-task Resolved Haibo Chen
          51.
          [atsv2 read acls] Add TimelineWriter#writeDomain Sub-task Resolved Rohith Sharma K S
          52.
          [Atsv2] Retrospect storing entities into sub application table from client perspective Sub-task Resolved Rohith Sharma K S
          53.
          Add table for storing timeline domain information Sub-task Resolved Unassigned
          54.
          ATSv2 REST API queries do not return results for uppercase application tags Sub-task Resolved Charan Hebri
          55.
          [atsv2 read acls] REST API to get domain info by id Sub-task Resolved Unassigned
          56.
          YarnClient should contact TimelineReader for application/attempt/container report Sub-task Resolved Abhishek Modi
          57.
          Support HBase 2.0.0-beta1 as ATSv2 backend Sub-task Resolved Unassigned
          58.
          Improve error message for invalid value in fields attribute Sub-task Resolved Abhishek Modi
          59.
          Give an informative message when incorrect format is used in ATSv2 filter attributes Sub-task Resolved Rohith Sharma K S
          60.
          ATS v2 returns invalid YARN_CONTAINER_ALLOCATED_HOST_HTTP_ADDRESS from NM Sub-task Resolved Rohith Sharma K S
          61.
          Incorrect HTTP status code returned by ATSv2 for non-whitelisted users Sub-task Resolved Rohith Sharma K S
          62.
          HTTPS Ats v2 api call fails with "bad HTTP parsed" Sub-task Resolved Charan Hebri
          63.
          Race condition when container events are published for KILLED applications Sub-task Resolved Rohith Sharma K S
          64.
          Adding JMX Metrics for Timeline Collector and Reader Sub-task Resolved Sushil Ks
          65.
          Provide Java client for fetching Yarn specific entities from TimelineReader Sub-task Resolved Abhishek Modi
          66.
          Document behavior of YARN-5742 Sub-task Resolved Suma Shivaprasad
          67.
          ApplicationCLI should have option to take clusterId Sub-task Resolved Rohith Sharma K S
          68.
          [atsv2] Restrict the number of elements held in timeline collector when backend is unreachable for async calls Sub-task Resolved Abhishek Modi
          69.
          HBaseTimelineWriterImpl sync writes has to avoid thread blocking if storage down Sub-task Resolved Prabhu Joseph
          70.
          Publish container killed, paused and resumed events to ATSv2. Sub-task Resolved Abhishek Modi
          71.
          ATS v2 Generic TimelineStorageMonitor Sub-task Resolved Prabhu Joseph
          72.
          Username splits won't help timelineservice.app_flow table Sub-task Resolved Prabhu Joseph
          73.
          Update document for ATS HBase Custom tablenames (-entityTableName) Sub-task Resolved Prabhu Joseph
          74.
          HBaseTimelineSchemaCreator skipExistingTable should honor custom hbase tablenames Sub-task Resolved Prabhu Joseph
          75.
          FlowActivity and FlowRun table prefix is wrong Sub-task Resolved Prabhu Joseph
          76.
          ATSV2 /apps/appId/entities/YARN_CONTAINER rest api does not show metrics Sub-task Resolved Prabhu Joseph
          77.
          RM Crashes with NPE at TimelineServiceV2Publisher.putEntity Sub-task Resolved Prabhu Joseph

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                vrushalic Vrushali C
              • Votes:
                0 Vote for this issue
                Watchers:
                18 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: