[HBASE-24318] Create-release scripts fixes and enhancements

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0-alpha-1, 2.3.0
• Fix Version/s: 3.0.0-alpha-1
• Component/s: create-release

Environment

Linux Docker container, or Mac OS X without Docker

Release Note

Release Managers will want to take note of two changes in the create-release tools:
- The names of the release steps are now `tag`, `publish-dist`, `publish-snapshot`, and `publish-release` (the latter two being mutually exclusive alternatives). `Publish-dist` was formerly just named `build`, which disguised the fact that it built AND published tarballs to https://dist.apache.org/repos/dist/release/ , and that publish-snapshot/publish-release do their own build before publishing maven artifacts to the Nexus repos.
- "Dry run" has become the _default_ behavior. Doing a dry run first decreases the likelihood of failure part-way through, which could result in inconsistent artifacts in the repos. To actually do the release, add the `-f` (force) option flag when you invoke `do-release-docker.sh` or `do-release.sh`. This is similar to `-f` usage in some Linux and git commands that have high cost for mistakes. This is a change from the former `-n` to turn ON dry run.

• Assignee: Matthew Foley
• Reporter: Matthew Foley
• Created: 04/May/20 17:14
• Updated: 13/May/20 17:50
• Resolved: 13/May/20 17:50

Description

The create-release tools are a set of scripts that promote best practice in making Apache releases, by automating most of the steps and running them under Docker for reliability. However, the current state of the scripts has many bugs and ambiguities.

The proposed PR cleans up the code and clarifies usage. It enables:

• Clear statement of the four steps, which are now called `tag`, `publish-dist`, `publish-snapshot`, and `publish-release` (the latter two being mutually exclusive alternatives).
• Ability to do the three tag-dist-release steps with a single command, or do any of the steps singly. (Running singly had bugs and unfulfilled dependencies before.)
• Ability to do a reliable and useful "dry run" of all steps or each step, and chain together the tag step with publish steps in a dry run.
• Ability to run any or all steps correctly in Docker or outside of Docker, on Linux and Mac.
• Cleaned up all `shellcheck` errors in the scripts, and removed ambiguities and redundancies in the many environment variables used.

In addition, the changes move the code toward being more general / less HBase-specific, so it can be run on any Apache project (while still accommodating HBase-specific features regarding how sub-projects are named and organized in Jira and release repos). In future I propose to take it further along that path, and move create-release into Yetus (recognizing that this create-release code has been passed between a couple other projects already).

These changes have NO IMPACT on HBase functionality.

Issue Links

• links to: GitHub Pull Request #1643
• links to: GitHub Pull Request #1643