Details
-
Bug
-
Status: Open
-
Trivial
-
Resolution: Unresolved
-
Impala 2.10.0
-
None
-
None
-
ghx-label-5
Description
I tried to do a test run of TPC-H SF 1 using the single_node_perf_run.py script, but run into a few problems:
- It isn't aware that those databases are specially named and don't include the scale factor in the DB name.
- When it tries to --load data for SF 1, it gets confused for a similar reason - the preload script generates data into /home/tarmstrong/Impala/incubator-impala/testdata/impala-data/tpch but the data load wants it to be in a slightly different directory:
Generating TPC-H data into /home/tarmstrong/Impala/incubator-impala/testdata/impala-data/tpch 0: jdbc:hive2://localhost:11050/default> LOAD DATA LOCAL INPATH '/home/tarmstrong/Impala/incubator-impala/testdata/impala-data/tpch1/lineitem' . . . . . . . . . . . . . . . . . . . .> OVERWRITE INTO TABLE tpch1.lineitem; going to print operations logs printed operations logs Getting log thread is interrupted, since query is done! Error: Error while compiling statement: FAILED: SemanticException Line 1:23 Invalid path ''/home/tarmstrong/Impala/incubator-impala/testdata/impala-data/tpch1/lineitem'': No files matching path file:/home/tarmstrong/Impala/incubator-impala/testdata/impala-data/tpch1/lineitem (state= 42000,code=40000) org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: SemanticException Line 1:23 Invalid path ''/home/tarmstrong/Impala/incubator-impala/testdata/impala-data/tpch1/lineitem'': No files matching path file:/home/tarmstrong/Impala/incubator-impala/test data/impala-data/tpch1/lineitem at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:241) at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:227) at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:255) at org.apache.hive.beeline.Commands.executeInternal(Commands.java:989) at org.apache.hive.beeline.Commands.execute(Commands.java:1177) at org.apache.hive.beeline.Commands.sql(Commands.java:1091) at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1176) at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:1010) at org.apache.hive.beeline.BeeLine.executeFile(BeeLine.java:987) at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:914) at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:518) at org.apache.hive.beeline.BeeLine.main(BeeLine.java:501) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) Caused by: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: SemanticException Line 1:23 Invalid path ''/home/tarmstrong/Impala/incubator-impala/testdata/impala-data/tpch1/lineitem'': No files matching path file:/home/tarmstrong/Impala/incubator- impala/testdata/impala-data/tpch1/lineitem at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:400) at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:187) at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:271) at org.apache.hive.service.cli.operation.Operation.run(Operation.java:337) at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:439) at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:416) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78) at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36) at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:421) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917) at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59) at com.sun.proxy.$Proxy19.executeStatementAsync(Unknown Source) at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:282) at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:501) at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1313) at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1298) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.hadoop.hive.ql.parse.SemanticException: Line 1:23 Invalid path ''/home/tarmstrong/Impala/incubator-impala/testdata/impala-data/tpch1/lineitem'': No files matching path file:/home/tarmstrong/Impala/incubator-impala/testdata/impala-data/tpch1/lineitem at org.apache.hadoop.hive.ql.parse.LoadSemanticAnalyzer.applyConstraints(LoadSemanticAnalyzer.java:139) at org.apache.hadoop.hive.ql.parse.LoadSemanticAnalyzer.analyzeInternal(LoadSemanticAnalyzer.java:218) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:223) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:558) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1356) at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1343) at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:185) ... 27 more Closing: 0: jdbc:hive2://localhost:11050/default;auth=none Error executing file from Hive: load-tpch-core-hive-generated.sql tarmstrong@tarmstrong-box2:~/Impala/incubator-impala$ /home/tarmstrong/Impala/incubator-impala/bin/load-data.py --workloads tpch --scale_factor 1 --table_formats text/none,parquet/none^C