[IMPALA-9365] Run hive-server2, YARN etc on non-HDFS test envs

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Infrastructure
• Target Version: Product Backlog
• Epic Color: ghx-label-4
• Assignee: Unassigned
• Reporter: Tim Armstrong
• Created: 07/Feb/20 19:39
• Updated: 08/Jun/21 02:13

Description

We don't start yarn unless HDFS is also running - https://github.com/apache/impala/blob/master/testdata/cluster/admin#L70.

We also don't start various other services, including hive server 2 https://github.com/apache/impala/blob/master/testdata/bin/run-all.sh#L79 .

This has been a persistent pain point for testing, because we can't load data and can't run many tests against those filesystems (see all the SkipIfS3 , etc annotations). Even worse, it can result in hard-to-debug failure modes like IMPALA-9345.

Issue Links

• causes: IMPALA-9608 Multiple query tests failure due to org.apache.hadoop.hive.ql.exec.tez.TezTask execution error
• is related to: IMPALA-9345 Hang in TestOrc::test_type_conversions when running create table in hive