[YARN-8023] REST API doesn't show new application

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Invalid
• Affects Version/s: 2.7.3
• Component/s: api

Environment

Release label:emr-5.5.0
Hadoop distribution:Amazon 2.7.3
Applications:Spark 2.1.0, Hive 2.1.1, Hue 3.12.0

• Assignee: Unassigned
• Reporter: Airton Sampaio de Sobral
• Created: 10/Mar/18 21:28
• Updated: 12/Mar/18 06:49
• Resolved: 12/Mar/18 06:49

Description

After killing an application using the HADOOP UI, and creating a new one with the same configuration, the brand new one doesn't appear on the API route:

/api/v1/applications?status=running

I've tried the applications path as well (/api/v1/applications) without success.

This happens randomly and it seems that after a really long time, the new instance appears on the API.

On the UI the new application instance appears, and it's working fine. (Print screen on attachment).

On the API it shows the last instance of the application as it's running, but it's dead by one hour!

/api/v1/applications?status=running

{
"id" : "application_1511385973584_0087",
"name" : "AdActionPaymentKafkaToJDBC",
"attempts" : [ {
"attemptId" : "1",
"startTime" : "2018-01-16T19:08:32.275GMT",
"endTime" : "1969-12-31T23:59:59.999GMT",
"lastUpdated" : "2018-01-16T19:08:34.016GMT",
"duration" : 0,
"sparkUser" : "hadoop",
"completed" : false,
"endTimeEpoch" : -1,
"startTimeEpoch" : 1516129712275,
"lastUpdatedEpoch" : 1516129714016
}

 

Update:

After two hours, the application appeared on the API response:

 

{
"id" : "application_1511385973584_0154",
"name" : "AdActionPaymentKafkaToJDBC",
"attempts" : [ {
"attemptId" : "1",
"startTime" : "2018-03-10T21:08:30.557GMT",
"endTime" : "1969-12-31T23:59:59.999GMT",
"lastUpdated" : "2018-03-10T21:08:32.310GMT",
"duration" : 0,
"sparkUser" : "hadoop",
"completed" : false,
"endTimeEpoch" : -1,
"startTimeEpoch" : 1520716110557,
"lastUpdatedEpoch" : 1520716112310
}