[SPARK-26420] [kubernetes] driver service id is not random

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.3.2
• Fix Version/s: 3.0.0
• Component/s: Kubernetes, Spark Core

Environment

Spark 2.3.2
Kubernetes 1.10.11

• Assignee: Marcelo Masiero Vanzin
• Reporter: Viktor Bogdanov
• Created: 20/Dec/18 09:17
• Updated: 17/May/20 18:25
• Resolved: 01/Mar/19 04:40

Description

I faced with a very rare situation

2018-12-20 04:34:46 WARN DriverServiceBootstrapStep:66 - Driver's hostname would preferably be one-job-41944f13918f3d779f7dc8cfea489791-driver-svc, but this is too long (must be <= 63 characters). Falling back to use spark-1545280486832-driver-svc as the driver service's name.

 2018-12-20 04:34:46 WARN DriverServiceBootstrapStep:66 - Driver's hostname would preferably be completely-different-job-017608e0359a3d7389667280f45ec562-driver-svc, but this is too long (must be <= 63 characters). Falling back to use spark-1545280486832-driver-svc as the driver service's name.

Two different spark-submits chouse the same name for driver service
This happened because service id is chosen by clock.getTimeMillis() function
code line

Issue Links

• links to: GitHub Pull Request #23805