[HDDS-6496] Improve CI checks

• Type: Improvement
• Status: In Progress
• Priority: Major
• Resolution: Unresolved
• Component/s: build, CI
• Assignee: Attila Doroszlai
• Reporter: Attila Doroszlai
• Created: 23/Mar/22 14:12
• Updated: 11/Dec/23 21:32

Description

Some ideas for improvement:

• compile (11) is not necessary for acceptance test changes, when compilation is necessary only to get the binary tarball for running the tests.
• basic checks are deterministic (except basic (unit) to some degree). Failures can only be fixed by code change, so we can restore fail-fast behavior.
• basic checks are also quick, so we can postpone starting integration until these finish.

Issue Links

• contains: HDDS-5730 Decrease run time of acceptance tests
• is a parent of: HDDS-9000 Speed up unit/integration tests
• relates to: HDDS-5626 Track and Address Flaky tests
• relates to: HDDS-8719 Improve thread names
• relates to: HDDS-6729 Migrate tests to JUnit5

Sub-Tasks

• Skip compile(11) for non-java changes: Resolved
• Reduce OzoneFS test combinations in ozonesecure: Resolved
• Create compat acceptance split: Resolved
• Avoid repeating some S3 tests for different bucket types: Resolved
• Further splits integration tests: Resolved
• Verify build from source release tarball in CI: Resolved
• Omit download progress in builds: Resolved
• Add CI to check pull request title: Resolved
• Increase timeout for basic check: Resolved
• Disallow ellipsis in PR title: Resolved
• Ignore pr_title_check for selective checks: Resolved
• Skip installing awscli if already present: Resolved
• Timed out tests missing from bundle: Resolved
• Separate scheduled CI from push/PR workflows: Resolved
• Suppress warning about long filenames in tar: Resolved
• Kubernetes check should fail fast if cluster cannot start: Resolved
• Acceptance tests need rebalancing: Resolved
• Reorder OM nodes in HA acceptance tests: Resolved
• Reduce S3 acceptance test setup time: Resolved
• Run HA secure tests from s3g container: Resolved
• Disable JaCoCo for PRs and in forks: Resolved
• Selective checks: handle change in junit.sh: Resolved
• Selective checks: handle change in ci.yaml: Resolved
• Increase timeout of CI build: Resolved
• Fail integration tests if RocksDB objects are leaked: Resolved
• Support multiple acceptance test scripts with custom config: Resolved
• Update compose README: Resolved
• Let run.sh start more than 3 datanodes: Resolved
• Selective checks: skip acceptance tests for compose doc changes: Resolved
• Test Hadoop compatibility in Ozone HA environment: Resolved
• Convert s3-haproxy to an add-on: Resolved
• Reduce ozone sh calls in robot tests: Resolved
• Reuse same Ozone cluster for Hadoop tests: Resolved
• Create separate check for native build: Resolved
• Ensure docker env is stopped: Resolved
• error validating kustomization.yaml: Resolved
• Keep separate Robot XMLs if rebot fails to combine them: Resolved
• Rebalance integration tests: Resolved
• Update instructions in dependency check: Resolved
• Add dedicated log file for each service in docker test: Resolved
• Error creating stack dump in S3 HA environment: Resolved
• Skip acceptance/kubernetes for license changes: Resolved
• Extract nodes required for SCM decommission test: Resolved
• Create separate acceptance split for cert rotation: Resolved
• Create a script to list all acceptance test splits: Resolved
• Merge unit and integration checks into a combined one: Resolved
• Require successful basic checks for long-running tests: Resolved
• Summarize failures of acceptance/kubernetes checks: Resolved
• Mark disabled tests using custom tag/category: Resolved
• Intermittent error while removing docker network: Resolved
• Handle empty unit check: Resolved
• Fail checks in Summary step instead of Test: Resolved
• Introduce a new workflow for testing intermittently failing unit tests.: Resolved
• Parameter for fail-fast behavior in flaky-test-check: Resolved
• Upload artifacts of unit/native check: Resolved
• Allow disabling JUnit timeouts: Resolved
• Increase timeout of unit check: Resolved
• Group output of check steps: Resolved
• Improve flaky-test-check job name.: Resolved
• Remove scheduled CI run: Resolved
• Skip unhealthy tests in repeated runs workflow: Resolved
• Redirect pre-test build output for integration check: Resolved
• Introduce FlakyTest category for flaky JUnit4 tests: Resolved
• Rebalance acceptance tests: Resolved
• Count total number of failures in flaky-test-check: Resolved
• Do not run all checks for draft PR touching CI files: Resolved
• Resolve dependabot build issues when updating npm packages: Resolved
• Create workflow for running acceptance test repeatedly: Resolved
• Skip coverage check for dependabot: Resolved
• Skip push build for dependabot: Resolved
• Use secure cluster for upgrade acceptance tests: Resolved
• Check that dependencies have acceptable license: Resolved
• Set descriptive title for upgrade Robot tests: Resolved
• Create stack dump if cannot remove docker network: Resolved
• Checkstyle check passing despite config error: Resolved
• Selective checks: skip tests for IntelliJ config change: Resolved
• Split parts of misc suite: Resolved
• Intermittent failure at hadoop-maven-plugin in compile check: Resolved
• Simplify Maven cache for dependencies in CI: Resolved
• Let junit.sh fail if no tests were matched in repeated run: Resolved
• Pre-build Ozone in flaky-test-check: Resolved
• Collect logs from cancelled integration test: Resolved
• Enable Surefire process checker: Resolved
• Remove validation of test method in flaky-test-check: Resolved
• Node.js 16 actions are deprecated: Resolved
• Increase timeout for compile check: Resolved
• Group dependabot PRs for frontend dependencies: Resolved
• Download zlib fails with 403 Forbidden in CI: Resolved
• Intermittent failure downloading from sourceware.org: Resolved
• Add check for dependencies included in Ozone FS shaded jar: Open
• No logs if pod stuck in PodInitializing: Open
• Consistently enable exit-on-error in test scripts: Open
• Test Ozone with Ratis "snapshot": Open
• Allow running nested test classes in flaky-test-check: Resolved
• Intermittent failure in compile (macos-12): Open
• Selective checks: skip tests for Dashboard changes: Resolved
• Schedule dependabot for weekend: Resolved
• Populate dependency cache in separate workflow: Resolved
• Fetch dependencies using actual build: Resolved
• Repeated tests stop if fork does not return: Patch Available
• Downgrade Surefire to 3.0.0-M4: Resolved
• Checkstyle summary excludes errors with "xml": Resolved
• Exclude proto3 classes from coverage: Resolved
• Rename unit check to native: Resolved
• Remove custom dependabot workflow: Resolved
• Only bump npm packages for security: Open