Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-6221

Entities missing from ATS when summary log file info got returned to the ATS before the domain log

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • None
    • 3.4.0, 3.2.3, 3.3.2, 3.1.5
    • yarn
    • None
    • Reviewed

    Description

      Events data missing for the following entities:

      REQUEST:

      curl -k --negotiate -u: http://<timeline_host>:8188/ws/v1/timeline/TEZ_APPLICATION_ATTEMPT/tez_appattempt_1487706062210_0012_000001
      

      RESPONSE:

      {"events":[],"entitytype":"TEZ_APPLICATION_ATTEMPT","entity":"tez_appattempt_1487706062210_0012_000001","starttime":1487711606077,"domain":"Tez_ATS_application_1487706062210_0012","relatedentities":{"TEZ_DAG_ID":["dag_1487706062210_0012_2","dag_1487706062210_0012_1"]},"primaryfilters":{},"otherinfo":{}}
      

      LOGS:

      Timeline Server log entry
      WARN  timeline.TimelineDataManager (TimelineDataManager.java:doPostEntities(366)) - Skip the timeline entity: { id: tez_application_1487706062210_0012, type: TEZ_APPLICATION }
      org.apache.hadoop.yarn.exceptions.YarnException: Domain information of the timeline entity { id: tez_application_1487706062210_0012, type: TEZ_APPLICATION } doesn't exist.
              at org.apache.hadoop.yarn.server.timeline.security.TimelineACLsManager.checkAccess(TimelineACLsManager.java:122)
              at org.apache.hadoop.yarn.server.timeline.TimelineDataManager.doPostEntities(TimelineDataManager.java:356)
              at org.apache.hadoop.yarn.server.timeline.TimelineDataManager.postEntities(TimelineDataManager.java:316)
              at org.apache.hadoop.yarn.server.timeline.EntityLogInfo.doParse(LogInfo.java:204)
              at org.apache.hadoop.yarn.server.timeline.LogInfo.parsePath(LogInfo.java:156)
              at org.apache.hadoop.yarn.server.timeline.LogInfo.parseForStore(LogInfo.java:113)
              at org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore$AppLogs.parseSummaryLogs(EntityGroupFSTimelineStore.java:682)
              at org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore$AppLogs.parseSummaryLogs(EntityGroupFSTimelineStore.java:657)
              at org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore$ActiveLogParser.run(EntityGroupFSTimelineStore.java:870)
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
              at java.util.concurrent.FutureTask.run(FutureTask.java:266)
              at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
              at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
              at java.lang.Thread.run(Thread.java:745)
      

      Attachments

        1. YARN-6221.02.patch
          3 kB
          Xiaomin Zhang
        2. YARN-6221.02.patch
          3 kB
          Xiaomin Zhang
        3. YARN-6221.branch-3.1.001.patch
          3 kB
          Xiaomin Zhang
        4. YARN-6221.branch-3.2.001.patch
          3 kB
          Xiaomin Zhang
        5. YARN-6221.branch-3.3.001.patch
          3 kB
          Xiaomin Zhang
        6. YARN-6221.branch-3.3.002.patch
          3 kB
          Xiaomin Zhang
        7. YARN-6221.patch
          3 kB
          Xiaomin Zhang
        8. YARN-6221.patch
          3 kB
          Xiaomin Zhang

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            ximz Xiaomin Zhang
            ssreenivasan Sushmitha Sreenivasan
            Votes:
            0 Vote for this issue
            Watchers:
            11 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment