[HIVE-21954] QTest: support for running qtests on various metastore DBs

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-1
• Component/s: Standalone Metastore, Testing Infrastructure
• Assignee: László Bodor
• Reporter: László Bodor
• Created: 04/Jul/19 07:20
• Updated: 17/Nov/22 08:49
• Resolved: 29/Nov/19 21:37

Description

In HIVE-21940, a postgres metastore related issue has been fixed, and a local reproduction has been provided.

export QTEST_LEAVE_FILES=true
docker kill metastore-test-postgres-install
docker rm metastore-test-postgres-install
cd standalone-metastore
mvn verify -DskipITests=false -Dit.test=ITestPostgres#install -Dtest=nosuch -Dmetastore.itest.no.stop.container=true
cd ..
mvn test -Dtest.output.overwrite=true -Pitests,hadoop-2 -pl itests/qtest -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q -Dhive.metastore.rawstore.impl=org.apache.hadoop.hive.metastore.ObjectStore

The problem with this solution is that data/conf/hive-site.xml has to be edited manually. My proposal is to introduce a property (-Dmetastore.db=postgres), which can take care of the parameters on the fly. 2 supported solutions could be:
1. simple parameters: -Dmetastore.db=postgres
In this case, tests depend on settings from ITestPostgres class (password, db, etc.)
2. verbose but flexible parameters: see hive-site.xml HIVE-21940's repro patch

In the long term, we should consider running this kind of tests in precommit phase, so maybe -Dmetastore.db=postgres could start metastore db automatically. Also we should consider running some qtests on various metastores. I would not pick randomly, but choose some "metastore-heavy" ones instead.

Issue Links

• causes: HIVE-22645 Jline can break bash terminal behavior
• is related to: HIVE-26005 Run selected qtest on different metastore backends