[BEAM-3249] Use Gradle to build/release project

• Type: Improvement
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Fix Version/s: 2.5.0
• Component/s: build-system, testing
• Assignee: Luke Cwik
• Reporter: Luke Cwik
• Created: 27/Nov/17 18:26
• Updated: 16/May/20 13:47
• Resolved: 17/Jul/18 17:51

Description

I have collected data by running several builds against master using Gradle and Maven without using Gradle's support for incremental builds.

Gradle (mins)
min: 25.04
max: 160.14
median: 45.78
average: 52.19
stdev: 30.80

Maven (mins)
min: 56.86
max: 216.55
median: 87.93
average: 109.10
stdev: 48.01

I excluded a few timeouts (240 mins) that happened during the Maven build from its numbers but we can see conclusively that Gradle is about twice as fast for the build when compared to Maven when run using Jenkins.

Original dev@ thread: https://lists.apache.org/thread.html/225dddcfc78f39bbb296a0d2bbef1caf37e17677c7e5573f0b6fe253@%3Cdev.beam.apache.org%3E
The data is available here https://docs.google.com/spreadsheets/d/1MHVjF-xoI49_NJqEQakUgnNIQ7Qbjzu8Y1q_h3dbF1M/edit?usp=sharing

Issue Links

• is a parent of: BEAM-3944 Convert beam_PerformanceTests_Python to use Gradle
• is a parent of: BEAM-3942 Update performance testing framework to use Gradle.
• is a parent of: BEAM-3943 Convert existing Java performance tests to use Gradle
• is a parent of: BEAM-4057 Ensure generated pom don't break consumers
• is a parent of: BEAM-4330 Sporadic ZipExceptions in tests with gradle and "isRelease"
• is related to: BEAM-3253 Gradle configurations do not declare correct inputs/outputs, requiring --rerun-tasks
• links to: GitHub Pull Request #4555
• links to: GitHub Pull Request #4812
• links to: GitHub Pull Request #4814
• links to: GitHub Pull Request #4830
• links to: GitHub Pull Request #4982
• links to: GitHub Pull Request #4992
• links to: GitHub Pull Request #5009
• links to: GitHub Pull Request #5062
• links to: GitHub Pull Request #5087
• links to: GitHub Pull Request #5098
• links to: GitHub Pull Request #5103
• links to: GitHub Pull Request #5107
• links to: GitHub Pull Request #5117
• links to: GitHub Pull Request #5858

Sub-Tasks

• Migrate ValidatesRunner Jenkins PostCommits to Gradle: Resolved
• Migrate Performance Tests Jenkins Jobs to use Gradle: Resolved
• Update contributors guide to discuss Gradle: Resolved
• Add Intellij hints to projects containing code generation tasks: Resolved
• Update release process to use Gradle: Resolved
• Add archetype testing/generation to existing GradleBuild PreCommit: Resolved
• Migrate Python Jenkins PreCommits to Gradle: Resolved
• Migrate Go Jenkins PreCommit to Gradle: Resolved
• On fresh checkout, failure of ./gradlew build due to no tox: Resolved
• On fresh checkout, failure of "./gradlew build" for Go due to "Cannot recognized package: common": Resolved
• Drop redundant "beam-" from project names in gradle: Resolved
• Update testing documentation with gradle info: Resolved
• Review and update the references of maven to gradle in the website: Resolved
• Review and update the references of maven to gradle in the source code: Resolved
• Build Go SDK container with Gradle: Resolved
• Build Go SDK examples with gradle: Resolved
• Gradle publish task should authenticate when run from jenkins: Resolved
• Update release guide to use gradle and not maven: Resolved
• Update release documentation for Gradle: Resolved
• Remove job_beam_Release_NightlySnapshot.groovy and add a nightly gradle jenkins job: Resolved
• Update performance testing framework to use Gradle.: Resolved
• Convert existing Java performance tests to use Gradle: Resolved
• Convert beam_PerformanceTests_Python to use Gradle: Open
• Update SDK container images and instructions to use Gradle: Resolved
• Use gradle and not maven in sdks/python/container/run_validatescontainer.sh: Resolved
• Maven pipeline jobs consistently failing: Resolved
• Apache Rat doesn't use .gitignore excludes when used with Apache Ant via Gradle: Resolved
• Migrate MavenInstall Jenkins PostCommits to Gradle: Resolved
• "No such file or directory" in beam_PreCommit_Python_GradleBuild: Resolved
• Ensure generated pom don't break consumers: Resolved
• Make sure Dataflow ValidatesRunner tests pass in Gradle: Resolved
• meta-info/poms missing in snapshots: Resolved
• Verify that the Build works on MacOS: Resolved
• Validate generated javadocs: Resolved
• Build times of IO tests with gradle take longer: Resolved
• Tune Gradle resource usage on Jenkins: Resolved
• Gradle publish -Prelease should create source .zip: Resolved
• Sporadic ZipExceptions in tests with gradle and "isRelease": Resolved
• Remove all mentions of "mvn" from javadoc: Open
• Improve gradle integration with IntelliJ: Resolved
• PostCommits are not running ValidatesRunner on DirectRunner: Resolved