[IMPALA-10458] load-data.sh fails to run Hive queries with IllegalArgumentException

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Infrastructure
• Epic Color: ghx-label-4
• Assignee: Unassigned
• Reporter: Quanlong Huang
• Created: 28/Jan/21 07:34
• Updated: 28/Jan/21 08:12

Description

Hit the error when running bin/load-data.sh in a local branch. Found details in logs/data_loading/sql/functional/load-functional-query-core-hive-generated-text-none-none.sql.log

!connect jdbc:hive2://localhost:11050/default;auth=none quanlong [passwd stripped]
Connecting to jdbc:hive2://localhost:11050/default;auth=none
21/01/28 15:20:25 [main]: WARN jdbc.HiveConnection: Failed to connect to localhost:11050
Error: Could not open client transport with JDBC Uri: jdbc:hive2://localhost:11050/default;auth=none: Failed to open new session: java.lang.IllegalArgumentException: Cannot modify mapreduce.cluster.local.dir at runtime. It is not in list of params that are allowed to be modified at runtime (state=08S01,code=0)
java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://localhost:11050/default;auth=none: Failed to open new session: java.lang.IllegalArgumentException: Cannot modify mapreduce.cluster.local.dir at runtime. It is not in list of params that are allowed to be modified at runtime 
        at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:349)
        at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
        at java.sql.DriverManager.getConnection(DriverManager.java:664)
        at java.sql.DriverManager.getConnection(DriverManager.java:208)
        at org.apache.hive.beeline.DatabaseConnection.connect(DatabaseConnection.java:145)
        at org.apache.hive.beeline.DatabaseConnection.getConnection(DatabaseConnection.java:209)
        at org.apache.hive.beeline.Commands.connect(Commands.java:1679)
        at org.apache.hive.beeline.Commands.connect(Commands.java:1574)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hive.beeline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:56)
        at org.apache.hive.beeline.BeeLine.execCommandWithPrefix(BeeLine.java:1456)
        at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1495)
        at org.apache.hive.beeline.BeeLine.connectUsingArgs(BeeLine.java:917)
        at org.apache.hive.beeline.BeeLine.initArgs(BeeLine.java:803)
        at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:1108)
        at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:1082)
        at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:546)
        at org.apache.hive.beeline.BeeLine.main(BeeLine.java:528)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
Caused by: org.apache.hive.service.cli.HiveSQLException: Failed to open new session: java.lang.IllegalArgumentException: Cannot modify mapreduce.cluster.local.dir at runtime. It is not in list of params that are allowed to be modified at runtime 
        at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:358)
        at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:349)
        at org.apache.hive.jdbc.HiveConnection.openSession(HiveConnection.java:872)
        at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:321)
        ... 26 more
Caused by: org.apache.hive.service.cli.HiveSQLException: Failed to open new session: java.lang.IllegalArgumentException: Cannot modify mapreduce.cluster.local.dir at runtime. It is not in list of params that are allowed to be modified at runtime 
        at org.apache.hive.service.cli.session.SessionManager.createSession(SessionManager.java:489)
        at org.apache.hive.service.cli.session.SessionManager.openSession(SessionManager.java:420)
        at org.apache.hive.service.cli.CLIService.openSession(CLIService.java:190)
        at org.apache.hive.service.cli.thrift.ThriftCLIService.getSessionHandle(ThriftCLIService.java:480)
        at org.apache.hive.service.cli.thrift.ThriftCLIService.OpenSession(ThriftCLIService.java:322)
        at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1497)
        at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1482)
        at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
        at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
        at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
        at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.hive.service.cli.HiveSQLException: java.lang.IllegalArgumentException: Cannot modify mapreduce.cluster.local.dir at runtime. It is not in list of params that are allowed to be modified at runtime
        at org.apache.hive.service.cli.session.HiveSessionImpl.configureSession(HiveSessionImpl.java:300)
        at org.apache.hive.service.cli.session.HiveSessionImpl.open(HiveSessionImpl.java:188)
        at org.apache.hive.service.cli.session.SessionManager.createSession(SessionManager.java:480)
        ... 13 more
Caused by: java.lang.IllegalArgumentException: Cannot modify mapreduce.cluster.local.dir at runtime. It is not in list of params that are allowed to be modified at runtime
        at org.apache.hadoop.hive.conf.HiveConf.verifyAndSet(HiveConf.java:5303)
        at org.apache.hadoop.hive.ql.processors.SetProcessor.setConf(SetProcessor.java:257)
        at org.apache.hadoop.hive.ql.processors.SetProcessor.setConf(SetProcessor.java:220)
        at org.apache.hadoop.hive.ql.processors.SetProcessor.setVariable(SetProcessor.java:191)
        at org.apache.hive.service.cli.session.HiveSessionImpl.configureSession(HiveSessionImpl.java:298)
        ... 15 more

Issue Links

• requires: IMPALA-10459 Remove workaround codes for MAPREDUCE-6441