[ARROW-2350] Shrink size of spark_integration Docker container

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 0.10.0
• Component/s: Continuous Integration
• Labels: docker pull-request-available spark
• External issue URL: https://github.com/apache/arrow/issues/18299
• Assignee: James Lamb
• Reporter: James Lamb
• Created: 24/Mar/18 05:14
• Updated: 11/Jan/23 07:20
• Resolved: 25/Mar/18 10:21

Description

I would like to propose a few changes to the spark_integration Dockerfile:

https://github.com/apache/arrow/tree/master/dev/spark_integration

The size of the resulting image can be reduced by making the following changes:

• consolidating all RUN commands into a single RUN layer (reducing the number of layers)
• running apt-get clean to clear out the package cache
• running conda clean --all to clear out cached package tarballs, abandoned package versions, and other build artifacts from all the libraries that are conda installed

I will be submitting a PR on GitHub shortly. Generating this issue first so I can tag my PR to it.

Issue Links

• links to: GitHub Pull Request #1787