Details
-
Improvement
-
Status: Resolved
-
Minor
-
Resolution: Fixed
-
2.3.0, 2.3.1
-
None
Description
Spark on Kubernetes provides logging of the pod/container status as a monitor of the job progress. However the logger just dumps the raw container status object leading to fairly unreadable output like so:
18/08/24 09:03:27 INFO LoggingPodStatusWatcherImpl: State changed, new state:
pod name: spark-groupby-1535101393784-driver
namespace: default
labels: spark-app-selector -> spark-47f7248122b9444b8d5fd3701028a1e8, spark-role -> driver
pod uid: 88de6467-a77c-11e8-b9da-a4bf0128b75b
creation time: 2018-08-24T09:03:14Z
service account name: spark
volumes: spark-local-dir-1, spark-conf-volume, spark-token-kjxkv
node name: tab-cmp4
start time: 2018-08-24T09:03:14Z
container images: rvesse/spark:latest
phase: Running
status: [ContainerStatus(containerID=docker://23ae58571f59505e837dca40455d0347fb90e9b88e2a2b145a38e2919fceb447, image=rvesse/spark:latest, imageID=docker-pullable://rvesse/spark@sha256:92abf0b718743d0f5a26068fc94ec42233db0493c55a8570dc8c851c62a4bc0a, lastState=ContainerState(running=null, terminated=null, waiting=null, additionalProperties={}), name=spark-kubernetes-driver, ready=true, restartCount=0, state=ContainerState(running=ContainerStateRunning(startedAt=Time(time=2018-08-24T09:03:26Z, additionalProperties={}), additionalProperties={}), terminated=null, waiting=null, additionalProperties={}), additionalProperties={})]
The LoggingPodStatusWatcher actually already includes code to nicely format this information but only invokes it at the end of the job:
18/08/24 09:04:07 INFO LoggingPodStatusWatcherImpl: Container final statuses:
Container name: spark-kubernetes-driver
Container image: rvesse/spark:latest
Container state: Terminated
Exit code: 0
It would be nice if we continually used the nice formatting throughout the logging.
We already have patched this on our internal fork and will upstream a fix shortly.
