[YETUS-723] Overhaul the docker support for 2018

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.9.0
• Component/s: Precommit

Release Note

<!-- markdown -->
This change represents a fairly large overhaul of precommit's Docker support in order to fix many outstanding bugs and feature requests. As a result, there are several incompatible changes from previous versions. Please see the precommit Docker documentation for additional information.

End Users: New Features
=======================

* Support for pulling a pre-built tag instead of building the base image, platforms, non-standard dockerd socket, and Docker-in-Docker.
* `COPY` and `ADD` support when building (requires Docker API version 18+).
* All Yetus built images are now labeled with `org.apache.yetus` to make them easier to find via docker inspect.
* Docker information in the vote table now properly includes a link to the Dockerfile if URLs, artifacting, etc, are turned on.
* Image cleanup should be more reliable.

End Users: Incompatible Changes
===============================

* The minimum Docker version is now version 17.xx. It was previously 6.xx.
* `--dockeronfail` is no longer supported.
* `ANT_ARGS`, `ANT_OPTS`, and `MAVEN_OPTS` are now honored. Before they were always overwritten.
* `yetus/tp-(number)` images are now removed after normal completion.
* Because of the addition of Docker-in-Docker support, Docker mode no longer defaults to privileged mode and the flag has been renamed to be consistent with the other flags. To enable privileged mode, the `--docker-privd` flag must be set to true.

Developers: New Features
========================

* `--docker-bash-debug` will run test-patch.sh with `bash -x` inside the container for extensive debugging.
* yetuslib now sports a new `yetus_file_to_array` function.

Developers: Incompatible Changes
================================

* test-patch.sh's `relative_dir` has been promoted to yetuslib as `yetus_relative_dir`
* The internal `ANT_ARGS` has been replaced with `YETUS_ANT_ARGS`
* Many changes to how parameters are passed between pre-Docker and post-Docker.

• Flags: Important
• Assignee: Allen Wittenauer
• Reporter: Allen Wittenauer
• Created: 20/Nov/18 05:55
• Updated: 31/Oct/20 22:13
• Resolved: 16/Dec/18 19:55

Description

Many core parts of the Docker support haven't been touched in 3 years. Since that time, we have learned many things about how people use Docker with test-patch and Docker itself has added many new features. It's time to overhaul the basic plug-in to take advantage of everything we've learned.

Issue Links

• causes: YETUS-737 perms are wrong on user_params.txt
• contains: YETUS-100 better document how to set JAVA_HOME in docker mode
• contains: YETUS-301 docker mode sometimes throws date errors
• contains: YETUS-454 docker cleanup should be more aggressive against tp- images
• contains: YETUS-197 personality in docker mode is wrong in the vote table
• contains: YETUS-230 A message in the vote table disappears if dockeronfail is fallback and the specified dockerfile doesn't exist
• contains: YETUS-4 Specifying additional build tool arguments to test-patch.sh
• contains: YETUS-185 Bash -x option in docker mode
• contains: YETUS-558 Provide ability to pass debug (-X) flag to maven
• contains: YETUS-631 Add new command line parameter to define custom mvn argument
• contains: YETUS-650 Switch Dockerfile.endstub to use a docker build arg
• contains: YETUS-702 simplify docker failure code
• contains: YETUS-203 include a link to the dockerfile used
• is related to: YETUS-976 MAVEN_OPTS in Dockerfile is not honored
• requires: YETUS-708 Clean up the source