[IMPALA-11232] Consider not adding some jars when starting HMS

• Type: Test
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: Impala 4.0.0
• Component/s: Infrastructure
• Epic Color: ghx-label-10
• Assignee: Fang-Yu Rao
• Reporter: Fang-Yu Rao
• Created: 07/Apr/22 21:56
• Updated: 15/Apr/22 16:57
• Resolved: 15/Apr/22 16:57

Description

Currently when we execute run-hive-server.sh with the "-with_ranger" flag, we add all the jar files under the folder of $RANGER_HOME/ews/webapp/WEB-INF/lib to 'HADOOP_CLASSPATH' before we call the shell script that starts the Hive service, i.e., bin/hive.

Recall that bin/hive would attempt to execute 'hadoop version' to determine the version of the underlying Hadoop distribution before starting the Hive Metastore and the HiveServer2. By default, this Hadoop command under the covers would add every path on 'HADOOP_CLASSPATH' to its own environment variable of 'CLASSPATH'. When there are too many paths added to 'CLASSPATH', the Hadoop command would fail with the error message of "Argument list too long." and hence the Hive service could not be started.

We execute run-hive-server.sh with the "-with_ranger" flag in order to run the end-to-end test of test_hive_with_ranger_setup. We should consider not adding some (or all) of the jar files to 'HADOOP_CLASSPATH' from the folder described if some (or all) of them are not required to run the end-to-end test so as to avoid the error mentioned above.