[SPARK-17803] Docker integration tests don't run with "Docker for Mac"

• Type: Dependency upgrade
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.0.1
• Fix Version/s: 2.0.2, 2.1.0
• Component/s: Tests

Environment

Mac OS X 10.10.5, Docker for Mac

• Assignee: Christian Kadner
• Reporter: Christian Kadner
• Created: 06/Oct/16 07:59
• Updated: 06/Oct/16 21:31
• Resolved: 06/Oct/16 21:30

Description

The docker-integration-tests can be run on Mac OS X with Docker Toolbox. However Docker Toolbox comes with a cumbersome setup (VirtualBox VM running a boot2docker Linux distro, tricky to do port mapping, ...).
A much preferable way to work with Docker on Mac OS X it is to use the Docker for Mac native application but Spark's docker-integration-tests don't run with Docker for Mac. The work-around to set DOCKER_HOST=unix:///var/run/docker.sock is not the most straight forward find.

We should upgrade the spotify docker-client dependency from 3.6.6 to 4.0.8 or later which supports "Docker for Mac" out of the box.

Issue Links

• links to: [Github] Pull Request #15378 (ckadner)