[PHOENIX-5730] Stabilize tests on master branch

• Type: Task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Labels: phoenix-hardening quality-improvement
• Assignee: Unassigned
• Reporter: Istvan Toth
• Created: 17/Feb/20 08:50
• Updated: 20/Apr/21 04:32
• Resolved: 20/Apr/21 04:32

Description

There are frequent integration test failures on the master branch, that have no apparent rhyme or reason.

Most of the failures seem to be related to timeouts, and performance issues.

The failures seem more frequent when testing against the HBase 2.1 and HBase 2.2 profiles.

Investigate the causes, and try to stabilise the tests.

Some ideas to start with:

• Try to copy the performance related minicluster settings from the integration tests in the 2.2 HBase branch 
• investigate the test execution times to see whether and where there is a correlation between execution time increases and HBase profiles.
• Investigate whether the failing tests have execution times that are close to the timeout values during successful runs

Issue Links

• is related to: PHOENIX-5779 SplitSystemCatalogIT tests fail with Multiple Regions error
• is related to: PHOENIX-5817 Prepatch script is a psycho killer
• is related to: PHOENIX-5962 Stabilize builds
• relates to: PHOENIX-5699 Investigate reducing chore intervals in MiniCluster to speed up tests
• relates to: PHOENIX-5769 Phoenix precommit Flapping HadoopQA Tests in master