Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Won't Fix
-
1.0.0
-
None
-
None
Description
scala> hql("create table t1(key int, value string)") ... scala> hql("create view v as select * from t1") ... scala> table("v") ... java.lang.NullPointerException at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:190) at org.apache.spark.sql.hive.MetastoreRelation.<init>(HiveMetastoreCatalog.scala:261) at org.apache.spark.sql.hive.HiveMetastoreCatalog.lookupRelation(HiveMetastoreCatalog.scala:66) at org.apache.spark.sql.hive.HiveContext$$anon$2.org$apache$spark$sql$catalyst$analysis$OverrideCatalog$$super$lookupRelation(HiveContext.scala:149) at org.apache.spark.sql.catalyst.analysis.OverrideCatalog$$anonfun$lookupRelation$3.apply(Catalog.scala:96) at org.apache.spark.sql.catalyst.analysis.OverrideCatalog$$anonfun$lookupRelation$3.apply(Catalog.scala:96) at scala.Option.getOrElse(Option.scala:120) at org.apache.spark.sql.catalyst.analysis.OverrideCatalog$class.lookupRelation(Catalog.scala:96) at org.apache.spark.sql.hive.HiveContext$$anon$2.lookupRelation(HiveContext.scala:155) at org.apache.spark.sql.SQLContext.table(SQLContext.scala:161) at .<init>(<console>:40) at .<clinit>(<console>) at .<init>(<console>:7) at .<clinit>(<console>) at $print(<console>) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:734) at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:983) at scala.tools.nsc.interpreter.IMain.loadAndRunReq$1(IMain.scala:573) at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:604) at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:568) at scala.tools.nsc.interpreter.ILoop.reallyInterpret$1(ILoop.scala:760) at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:805) at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:717) at scala.tools.nsc.interpreter.ILoop.processLine$1(ILoop.scala:581) at scala.tools.nsc.interpreter.ILoop.innerLoop$1(ILoop.scala:588) at scala.tools.nsc.interpreter.ILoop.loop(ILoop.scala:591) at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:882) at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:837) at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:837) at scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135) at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:837) at scala.tools.nsc.interpreter.ILoop.main(ILoop.scala:904) at xsbt.ConsoleInterface.run(ConsoleInterface.scala:69) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at sbt.compiler.AnalyzingCompiler.call(AnalyzingCompiler.scala:102) at sbt.compiler.AnalyzingCompiler.console(AnalyzingCompiler.scala:77) at sbt.Console.sbt$Console$$console0$1(Console.scala:23) at sbt.Console$$anonfun$apply$2$$anonfun$apply$1.apply$mcV$sp(Console.scala:24) at sbt.Console$$anonfun$apply$2$$anonfun$apply$1.apply(Console.scala:24) at sbt.Console$$anonfun$apply$2$$anonfun$apply$1.apply(Console.scala:24) at sbt.Logger$$anon$4.apply(Logger.scala:90) at sbt.TrapExit$App.run(TrapExit.scala:244) at java.lang.Thread.run(Thread.java:745)
This issue can also be triggered by console version of the join_view test case of the HiveCompatibilitySuite:
== Console version of this test == val q0 = hql("""drop table invites"""); q0.collect() val q1 = hql("""drop table invites2"""); q1.collect() val q2 = hql("""create table invites (foo int, bar string) partitioned by (ds string)"""); q2.collect() val q3 = hql("""create table invites2 (foo int, bar string) partitioned by (ds string)"""); q3.collect() val q4 = hql("""set hive.mapred.mode=strict"""); q4.collect() val q5 = hql("""-- test join views: see HIVE-1989 create view v as select invites.bar, invites2.foo, invites2.ds from invites join invites2 on invites.ds=invites2.ds"""); q5.collect() val q6 = hql("""explain select * from v where ds='2011-09-01'"""); q6.collect() val q7 = hql("""drop view v"""); q7.collect() val q8 = hql("""drop table invites"""); q8.collect() val q9 = hql("""drop table invites2"""); q9.collect()
When running inside HiveCompatibilitySuite, the exception is eaten up by SQLContext.stringOrError.
When attacking SPARK-2094, we added physical plans for DDL and commands, no longer taking the stringOnError route, and the exception is thrown.
Attachments
Issue Links
- relates to
-
SPARK-4239 support view in HiveQL
- Resolved