[YUNIKORN-766] driver/executor get hang occasionally if the applicationId is defined

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.12.1
• Component/s: shim - kubernetes
• Labels: pull-request-available
• Target Version: 0.12.1
• Assignee: Chia-Ping Tsai
• Reporter: Chia-Ping Tsai
• Created: 29/Jul/21 16:18
• Updated: 17/Feb/22 15:29
• Resolved: 30/Jul/21 05:09

Description

The code used to get applicationId is shown below.

// application ID can be defined in labels
for name, value := range pod.Labels {
// application ID can be defined as a label
if name == constants.LabelApplicationID

{ return value, nil }

// if a pod for spark already provided appID, reuse it
if name == constants.SparkLabelAppID { return value, nil }

}

The iteration order is random so it could get application id from 'spark-app-selector' rather than `applicationId` (even though `applicationId` is defined by users)

Issue Links

• causes: YUNIKORN-946 Accounting resources for deleted executor pods
• links to: GitHub Pull Request #288