Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
Impala 2.6.0, impala 2.5.1, Impala 2.7.0
-
None
Description
A pytest bug (see IMPALA-2943) has made it so that most custom cluster tests don't get run against release builds or the S3 filesystem. The problem stems from a class of pytest bugs involving marking:
https://github.com/pytest-dev/pytest/issues/568
https://github.com/pytest-dev/pytest/issues/863
...and several others.
While this bug could be lurking anywhere the following are true:
1. a base class has a skipif mark defined, truth notwithstanding
2. a child class has a skipif mark that is true for the run
3. other child classes of the same base exist
..it's particularly bad in the custom cluster tests because the custom cluster base test class has a class-wide mark, and tests found early in the collection cycle have skip marks that are true on release or S3 jobs. In particular:
@SkipIfNotDebugBuild.debug_only
class TestAllocFail(CustomClusterTestSuite):
@SkipIfS3.caching
class TestHdfsFdCaching(CustomClusterTestSuite):
You can see the evidence here, among others:
============================= test session starts ============================== platform linux2 -- Python 2.6.6 -- py-1.4.30 -- pytest-2.7.2 rootdir: /data/jenkins/workspace/impala-umbrella-build-and-test/repos, inifile: plugins: random, xdist collected 52 items custom_cluster/test_admission_controller.py ..... custom_cluster/test_alloc_fail.py ss custom_cluster/test_breakpad.py ssss custom_cluster/test_delegation.py sss custom_cluster/test_exchange_delays.py ss custom_cluster/test_hdfs_fd_caching.py s custom_cluster/test_hive_parquet_timestamp_conversion.py ss custom_cluster/test_insert_behaviour.py ss custom_cluster/test_kudu_not_available.py s custom_cluster/test_legacy_joins_aggs.py s custom_cluster/test_parquet_max_page_header.py s custom_cluster/test_permanent_udfs.py sss custom_cluster/test_query_expiration.py sss custom_cluster/test_redaction.py ssss custom_cluster/test_s3a_access.py s custom_cluster/test_scratch_disk.py ssss custom_cluster/test_session_expiration.py s custom_cluster/test_spilling.py ssss authorization/test_authorization.py ss authorization/test_grant_revoke.py s ../../Impala-auxiliary-tests/tests/aux_custom_cluster_tests/test_ldap.py sssss generated xml file: /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/logs/custom_cluster_tests/results/TEST-impala-custom-cluster.xml ============== 5 passed, 47 skipped, 3 warnings in 205.73 seconds ==============