[FLINK-29708] Enrich Flink Kubernetes Operator CRD error field

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: kubernetes-operator-1.3.0
• Fix Version/s: kubernetes-operator-1.3.0
• Component/s: Kubernetes Operator
• Labels: pull-request-available
• Assignee: Daren Wong
• Reporter: Daren Wong
• Created: 20/Oct/22 17:59
• Updated: 27/Oct/22 13:59
• Resolved: 27/Oct/22 13:59

Description

Problem Statement:

FlinkDeployment and FlinkSessionJob CRD has a CommonStatus error field of String type. Currently, this field stores various errors such as:

• CR validation error
• Missing SessionJob error/ Missing JobManager deployment error
• Unknown Job error
• DeploymentFailedException
• ReconciliationError such as RestClientException from Flink Internal such as FlinkRest and FlinkRuntime

It is insufficient to store each error simply as string only. We need to include some exception metadata to help operator handle this error accordingly. For example, it is very useful to know the HttpResponseStatus code from RestClientException.

Proposed Solution:

• The error field should store a JSON with exception metadata. For example:

{    
  "type": "JobManagerNotFoundException",    
  "message": "JobManager with leadership ID: 1234 was not found",    
  "stackTrace": "JobManager lost connection at ....",
  "additionalMetadata": {     
    "httpResponseCode": "400"
  },
  "throwableList": [
    {
      "type": "FlinkRuntimeException",
      "message": "other exception"
    },
    ....
  ]
} 

• The stackTrace field can be enabled or disabled via spec change.

Issue Links

• links to: GitHub Pull Request #409