[BIGTOP-2947] Project Frontier: Bigtop Integration Test Framework 2.0

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.2.1
• Fix Version/s: 1.4.0
• Assignee: Evans Ye
• Reporter: Evans Ye
• Created: 28/Nov/17 14:32
• Updated: 30/Mar/19 18:21
• Resolved: 30/Mar/19 18:21

Description

The Project Frontier focuses on extending and harden the feature that Bigtop was originally designed for: Build Hadoop Distributions. Bigtop can only produce high quality distributions if working with upstream projects closely to solve integration problems across multiple Hadoop ecosystem projects.

Ref: https://www.odpi.org/blog/2017/10/17/checking-in-apache-bigtop-test-drive-grant-recipients-work-in-full-swing

Sub-Tasks

• Add gradle task which leverage bigtop-ci/build.sh to build packages: Resolved
• bigtop/bigtop-tests/smoke-tests/run_itest.sh is broken: Resolved
• Add pre requisite check task to check docker environment: Resolved
• Switch to gradle XXX-pkg-ind feature developed in BIGTOP-2949 for Bigtop CI: Resolved
• Document the new feature added in BIGTOP-2949: Resolved
• Add example CI pipeline to build end-to-end from a commit to smoke test: Resolved
• Extend Provisioner CI matrix to run smoke tests: Resolved
• Update Smoke Test CI matrix against Bigtop master Distros: Resolved
• Add AARCH64 in Provisioner CI matrix: Resolved
• Support configure-nexus for gradle COMPONENT-pkg-ind: Resolved
• Add gradle repo-ind: Resolved
• Release number is not honored by the build system: Resolved
• Better user expereince when calling docker-hadoop.sh -d with no cluster running: Resolved
• Nexus proxy cache configuration is broken: Resolved
• Error out when configure-nexus encounter any issue: Resolved
• Support to enable Nexus proxy for Docker Provisioner: Resolved
• [Provisioner] Raise proper exit code and error messages when provisioning failed: Resolved
• Switch to docker cp based solution to build inside containers: Resolved
• [Provisioner] Failed to bootstrap provisioning environment on Debian/Ubuntu: Resolved
• Setup manual smoke test for Pull Requests: Resolved
• Failed to start Ambari server and agent on Fedora 26: Resolved
• [Test] Fix test specification with maven failsafe plugin: Resolved
• Failed to deploy Alluxio due to Puppet compatibility issue: Resolved
• [Provisioner] Treat both exit code 0 and 2 as success for Puppet execution: Resolved
• Upgrade to Puppet 5.X: Resolved
• Update Kerberos module to comply with Puppet's syntax: Resolved
• Nexus is always enabled for ./gradlew XXX-ind: Resolved
• Enable local repo is not working on yum systems: Resolved
• Loop and wait for daemon to be started up for Alluxio: Resolved
• [Provisioner] Support to specify components and smoke-tests when launching Docker Provisioner: Resolved
• Fix bugs and improve usability of COMPONENT-pkg-ind: Resolved
• ./gradlew repo-ind failed on Bigtop build slaves: Resolved
• [Build] Support to specify commit SHA1 when building packages: Resolved
• [Build] Support non X86 archs for COMPONENT-pkg-ind: Resolved
• [Build] Exit w/o container left dangling when running docker in gradle wrapper: Resolved
• Connection refused by www-us.apache.org mirror site: Resolved
• Gradle 5 incompatible issue: Resolved
• Build as non-root for COMPONENT-pkg-ind: Resolved
• Helper message and documentation of new XXX-ind features: Resolved
• Allow to specify more properties/options at command line for docker provisioner: Resolved
• Add REPO and MEMORY as options for docker-provisioner: Resolved
• Add pkgs-ind to package all artifacts inside docker: Resolved
• [Docker] Treat both exit code 0 and 2 as success for Puppet execution: Resolved
• Can not run smoke test inside docker-provisioner: Resolved
• Bugfix Integration test framework 2.0 and documentation: Resolved