[YARN-9418] ATSV2 /apps/appId/entities/YARN_CONTAINER rest api does not show metrics

• Type: Sub-task
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 3.2.0
• Fix Version/s: 3.3.0, 3.2.1
• Component/s: ATSv2
• Assignee: Prabhu Joseph
• Reporter: Prabhu Joseph
• Created: 27/Mar/19 18:28
• Updated: 23/Oct/19 16:15
• Resolved: 01/Apr/19 18:10

Description

ATSV2 entities rest api does not show the metrics

[hbase@yarn-ats-3 centos]$ curl -s "http://yarn-ats-3:8198/ws/v2/timeline/apps/application_1553685341603_0006/entities/YARN_CONTAINER/container_e18_1553685341603_0006_01_000001?user.name=hbase&fields=METRICS" | jq .
{
"metrics": [],
"events": [],
"createdtime": 1553695002014,
"idprefix": 0,
"type": "YARN_CONTAINER",
"id": "container_e18_1553685341603_0006_01_000001",
"info": {
"UID": "ats!application_1553685341603_0006!YARN_CONTAINER!0!container_e18_1553685341603_0006_01_000001",
"FROM_ID": "ats!hbase!QuasiMonteCarlo!1553695001394!application_1553685341603_0006!YARN_CONTAINER!0!container_e18_1553685341603_0006_01_000001"
},
"configs": {},
"isrelatedto": {},
"relatesto": {}
}

NodeManager puts YARN_CONTAINER entities with CPU and MEMORY metrics but this is not shown in above output. Found NM container entities are set with entityIdPrefix as inverted container starttime whereas RM container entities are set with default 0. TimelineReader fetches only RM container entries.

Confirmed with setting NM container entities entityIdPrefix to 0 same as RM (for testing purpose) and found metrics are shown.

"metrics": [
{
"type": "SINGLE_VALUE",
"id": "MEMORY",
"aggregationOp": "NOP",
"values": {
"1553774981355": 490430464
}
},
{
"type": "SINGLE_VALUE",
"id": "CPU",
"aggregationOp": "NOP",
"values": {
"1553774981355": 5
}
}
]

 

Issue Links

• relates to: YARN-5792 adopt the id prefix for YARN, MR, and DS entities
• relates to: YARN-9382 Publish container killed, paused and resumed events to ATSv2.
• relates to: YARN-9452 Fix TestDistributedShell and TestTimelineAuthFilterForV2 failures