[IMPALA-11724] Ozone tests fail after CDP build number update

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 4.2.0
• Fix Version/s: Impala 4.2.0
• Labels: ozone
• Epic Color: ghx-label-14
• Assignee: Michael Smith
• Reporter: Michael Smith
• Created: 14/Nov/22 21:25
• Updated: 29/Nov/22 10:52
• Resolved: 16/Nov/22 22:16

Description

After IMPALA-11524, the test custom_cluster.test_custom_hive_configs.TestCustomHiveConfigs.test_ctas_read_write_consistence fails when run with Ozone because the hive server fails to come up

common/custom_cluster_test_suite.py:150: in setup_method
    self._start_hive_service(method.func_dict[HIVE_CONF_DIR])
common/custom_cluster_test_suite.py:248: in _start_hive_service
    raise RuntimeError("Unable to start Hive")
E   RuntimeError: Unable to start Hive

Hive server logs show

java.lang.RuntimeException: Error applying authorization policy on hive configuration: VOLUME_NOT_FOUND org.apache.hadoop.ozone.om.exceptions.OMException: Volume tmp is not found
	at org.apache.hive.service.cli.CLIService.init(CLIService.java:121) ~[hive-service-3.1.3000.7.2.16.0-233.jar:3.1.3000.7.2.16.0-233]
	at org.apache.hive.service.CompositeService.init(CompositeService.java:59) ~[hive-service-3.1.3000.7.2.16.0-233.jar:3.1.3000.7.2.16.0-233]
	at org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:247) ~[hive-service-3.1.3000.7.2.16.0-233.jar:3.1.3000.7.2.16.0-233]
	at org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:1152) [hive-service-3.1.3000.7.2.16.0-233.jar:3.1.3000.7.2.16.0-233]
	at org.apache.hive.service.server.HiveServer2.access$1400(HiveServer2.java:144) [hive-service-3.1.3000.7.2.16.0-233.jar:3.1.3000.7.2.16.0-233]
	at org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:1454) [hive-service-3.1.3000.7.2.16.0-233.jar:3.1.3000.7.2.16.0-233]
	at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:1267) [hive-service-3.1.3000.7.2.16.0-233.jar:3.1.3000.7.2.16.0-233]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_352]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_352]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_352]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_352]
	at org.apache.hadoop.util.RunJar.run(RunJar.java:318) [hadoop-common-3.1.1.7.2.16.0-233.jar:?]
	at org.apache.hadoop.util.RunJar.main(RunJar.java:232) [hadoop-common-3.1.1.7.2.16.0-233.jar:?]
Caused by: java.lang.RuntimeException: VOLUME_NOT_FOUND org.apache.hadoop.ozone.om.exceptions.OMException: Volume tmp is not found
	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:755) ~[hive-exec-3.1.3000.7.2.16.0-233.jar:3.1.3000.7.2.16.0-233]
	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:690) ~[hive-exec-3.1.3000.7.2.16.0-233.jar:3.1.3000.7.2.16.0-233]
	at org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:133) ~[hive-service-3.1.3000.7.2.16.0-233.jar:3.1.3000.7.2.16.0-233]
	at org.apache.hive.service.cli.CLIService.init(CLIService.java:118) ~[hive-service-3.1.3000.7.2.16.0-233.jar:3.1.3000.7.2.16.0-233]
	... 12 more
Caused by: org.apache.hadoop.ozone.om.exceptions.OMException: Volume tmp is not found
	at org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.handleError(OzoneManagerProtocolClientSideTranslatorPB.java:696) ~[ozone-filesystem-hadoop3-1.3.0.7.2.16.0-233.jar:?]
	at org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.getVolumeInfo(OzoneManagerProtocolClientSideTranslatorPB.java:404) ~[ozone-filesystem-hadoop3-1.3.0.7.2.16.0-233.jar:?]
	at org.apache.hadoop.ozone.client.rpc.RpcClient.getVolumeDetails(RpcClient.java:503) ~[ozone-filesystem-hadoop3-1.3.0.7.2.16.0-233.jar:?]
	at org.apache.hadoop.fs.ozone.BasicRootedOzoneClientAdapterImpl.getBucket(BasicRootedOzoneClientAdapterImpl.java:294) ~[ozone-filesystem-hadoop3-1.3.0.7.2.16.0-233.jar:?]
	at org.apache.hadoop.fs.ozone.BasicRootedOzoneClientAdapterImpl.getBucket(BasicRootedOzoneClientAdapterImpl.java:239) ~[ozone-filesystem-hadoop3-1.3.0.7.2.16.0-233.jar:?]
	at org.apache.hadoop.fs.ozone.BasicRootedOzoneClientAdapterImpl.createDirectory(BasicRootedOzoneClientAdapterImpl.java:469) ~[ozone-filesystem-hadoop3-1.3.0.7.2.16.0-233.jar:?]
	at org.apache.hadoop.fs.ozone.BasicRootedOzoneFileSystem.mkdir(BasicRootedOzoneFileSystem.java:824) ~[ozone-filesystem-hadoop3-1.3.0.7.2.16.0-233.jar:?]
	at org.apache.hadoop.fs.ozone.BasicRootedOzoneFileSystem.mkdirs(BasicRootedOzoneFileSystem.java:835) ~[ozone-filesystem-hadoop3-1.3.0.7.2.16.0-233.jar:?]
	at org.apache.hadoop.hive.ql.exec.Utilities.createDirsWithPermission(Utilities.java:3991) ~[hive-exec-3.1.3000.7.2.16.0-233.jar:3.1.3000.7.2.16.0-233]
	at org.apache.hadoop.hive.ql.exec.Utilities.ensurePathIsWritable(Utilities.java:5030) ~[hive-exec-3.1.3000.7.2.16.0-233.jar:3.1.3000.7.2.16.0-233]
	at org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:878) ~[hive-exec-3.1.3000.7.2.16.0-233.jar:3.1.3000.7.2.16.0-233]
	at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:819) ~[hive-exec-3.1.3000.7.2.16.0-233.jar:3.1.3000.7.2.16.0-233]
	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:731) ~[hive-exec-3.1.3000.7.2.16.0-233.jar:3.1.3000.7.2.16.0-233]
	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:690) ~[hive-exec-3.1.3000.7.2.16.0-233.jar:3.1.3000.7.2.16.0-233]
	at org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:133) ~[hive-service-3.1.3000.7.2.16.0-233.jar:3.1.3000.7.2.16.0-233]
	at org.apache.hive.service.cli.CLIService.init(CLIService.java:118) ~[hive-service-3.1.3000.7.2.16.0-233.jar:3.1.3000.7.2.16.0-233]
	... 12 more

It looks like the build of Hive being used here requires a pre-release version of Ozone.

Issue Links

• is caused by: IMPALA-11524 Remove workaround when HADOOP-18410 is fixed