[SPARK-26025] Docker image build on dev builds is slow

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.0.0
• Component/s: Kubernetes, Spark Core
• Assignee: Marcelo Masiero Vanzin
• Reporter: Marcelo Masiero Vanzin
• Created: 12/Nov/18 22:21
• Updated: 17/May/20 18:25
• Resolved: 27/Nov/18 17:09

Description

When building a docker image, the docker tool uploads the contents of the current directory - the "build context" - to the docker daemon.

On a dev build, that can be huge; with intermediate artifacts and test logs, it can easily be a few gigabytes. And that copy happens for each one of the 3 images build by the docker tool script.

It would be nice to make that faster to speed up iterative development.

Issue Links

• is duplicated by: SPARK-24219 Improve the docker build script to avoid copying everything in example
• links to: [Github] Pull Request #23019 (vanzin)
• links to: [Github] Pull Request #23019 (vanzin)