[HIVE-27504] Automate inclusion of 3-party licenses in binary distribution

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0-beta-1
• Labels: pull-request-available
• Assignee: Stamatis Zampetakis
• Reporter: Stamatis Zampetakis
• Created: 13/Jul/23 14:49
• Updated: 15/Aug/23 06:33
• Resolved: 28/Jul/23 23:14

Description

Manually downloading and including licenses of bundled dependencies in the binary distribution (as it is done currently under [binary-package-licenses directory|
https://github.com/apache/hive/tree/bd02abc9eba3aa0cd3701b26d9169130551a12ef/binary-package-licenses]) is error-prone and time consuming.

Moreover, dependencies and their versions change rapidly and it is hard to keep the LICENSE documentation correct and up-to-date. At the moment, the binary-package-licenses directory is stale; some bundled dependencies are missing, others do not longer exist, and versions are not mentioned anywhere so we cannot know if the license has changed.

Furthermore, every dependency change (add/remove/version bump) potentially requires commits and modifications to the source code.

The goal of this ticket is to automate the download and inclusion of 3rd-party licenses in the binary distribution to minimize/eliminate the need for human intervention. As part of the automation the following issues should be fixed as well:

1. LICENSE/NOTICE should reflect exactly what lies inside each package distribution (source & binary); ideally no more and definitely not less.
2. LICENSES for bundled dependencies should be referenced from the main LICENSE file not from NOTICE (or other places) as it is right now.
3. For bundled dependencies their versions should be included cause different version may use different licenses and it should be clear which version is bundled.

Issue Links

• links to: GitHub Pull Request #4484