[TEZ-4290] Dockerfile improvements

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.10.1, 0.9.3
• Assignee: László Bodor
• Reporter: László Bodor
• Created: 12/Feb/21 12:02
• Updated: 12/Feb/21 15:36
• Resolved: 12/Feb/21 15:35

Description

Given these improvement ideas: https://github.com/aw-was-here/tez/commit/bead3a21e68ecf4a75d64d8c08b9f8fa0f98cf51
thanks aw

Dockerfile changes could be easily tested, because it can be built locally (as discovered on TEZ-4283):

docker build -f ./build-tools/docker/Dockerfile ./build-tools/docker/

Moreover, it's applied on PR level, so a full precommit test (including all modules) could let us validate the correctness of the image changes.

So, I'm suspecting that Dockerfile changes can come in advance, before introducing Jenkinsfile, which is a bit harder to test (I guess).

according to aw's comments:

  - Update to an image that won't be unsupported in a year
  - Remove a bunch of stuff that Tez doesn't use
  - Add support for a bunch of stuff that Yetus needs to test everything (however, keeping FindBugs over SpotBugs)
  - Make it Docker BuildKit compatible to make it build in parallel
  - Support both OpenJDK 8 and OpenJDK 11
  - Switch to python3 because python2 is effectively dead in January with pip changes
  - Remove the interactive bits of the Dockerfile since nothing appears to be using them

Issue Links

• links to: GitHub Pull Request #111