[YARN-9403] GET /apps/{appid}/entities/YARN_APPLICATION accesses application table instead of entity table

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.2.0
• Fix Version/s: 3.4.0
• Component/s: ATSv2
• Labels: pull-request-available
• Assignee: Ashutosh Gupta
• Reporter: Prabhu Joseph
• Created: 22/Mar/19 13:46
• Updated: 02/Jul/22 16:30
• Resolved: 02/Jul/22 16:30

Description

"GET /apps/{appid}/entities/YARN_APPLICATION"

accesses application table instead of entity table. As per the doc, With this API, you can query generic entities identified by cluster ID, application ID and per-framework entity type. But it also provides all the apps when entityType is set to YARN_APPLICATION. It should only access Entity Table through GenericEntityReader.

Wrong Output: With YARN_APPLICATION entityType, all applications listed from application tables.

[hbase@yarn-ats-3 centos]$ curl -s "http://yarn-ats-3:8198/ws/v2/timeline/apps/application_1553258815132_0002/entities/YARN_APPLICATION?user.name=hbase&userid=hbase&flowname=word%20count" | jq .
[
  {
    "metrics": [],
    "events": [],
    "createdtime": 1553258922721,
    "idprefix": 0,
    "isrelatedto": {},
    "relatesto": {},
    "info": {
      "UID": "ats!application_1553258815132_0002",
      "FROM_ID": "ats!hbase!word count!1553258922721!application_1553258815132_0002"
    },
    "configs": {},
    "type": "YARN_APPLICATION",
    "id": "application_1553258815132_0002"
  },
  {
    "metrics": [],
    "events": [],
    "createdtime": 1553258825918,
    "idprefix": 0,
    "isrelatedto": {},
    "relatesto": {},
    "info": {
      "UID": "ats!application_1553258815132_0001",
      "FROM_ID": "ats!hbase!word count!1553258825918!application_1553258815132_0001"
    },
    "configs": {},
    "type": "YARN_APPLICATION",
    "id": "application_1553258815132_0001"
  }
]

Right Output: With correct entity type (MAPREDUCE_JOB) it accesses entity table for given applicationId and entityType.

[hbase@yarn-ats-3 centos]$ curl -s "http://yarn-ats-3:8198/ws/v2/timeline/apps/application_1553258815132_0002/entities/MAPREDUCE_JOB?user.name=hbase&userid=hbase&flowname=word%20count" | jq .
[
  {
    "metrics": [],
    "events": [],
    "createdtime": 1553258926667,
    "idprefix": 0,
    "isrelatedto": {},
    "relatesto": {},
    "info": {
      "UID": "ats!application_1553258815132_0002!MAPREDUCE_JOB!0!job_1553258815132_0002",
      "FROM_ID": "ats!hbase!word count!1553258922721!application_1553258815132_0002!MAPREDUCE_JOB!0!job_1553258815132_0002"
    },
    "configs": {},
    "type": "MAPREDUCE_JOB",
    "id": "job_1553258815132_0002"
  }
]

Flow Activity and Flow Run tables can also be accessed using similar way.

GET /apps/{appid}/entities/YARN_FLOW_ACTIVITY
GET /apps/{appid}/entities/YARN_FLOW_RUN

Issue Links

• links to: GitHub Pull Request #4516