Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-12982

SQLContext: temporary table registration does not accept valid identifier

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Resolved
    • Affects Version/s: 1.6.0
    • Fix Version/s: 2.0.0
    • Component/s: SQL
    • Labels:

      Description

      We have encountered very strange behavior of SparkSQL temporary table registration.
      What identifiers for temporary table should be valid?
      Alphanumerical + '_' with at least one non-digit?

      Valid identifiers:
      df
      674123a
      674123_
      a0e97c59_4445_479d_a7ef_d770e3874123
      1ae97c59_4445_479d_a7ef_d770e3874123
      Invalid identifier:
      10e97c59_4445_479d_a7ef_d770e3874123

      Stack trace:

      java.lang.RuntimeException: [1.1] failure: identifier expected
      
      10e97c59_4445_479d_a7ef_d770e3874123
      ^
      	at scala.sys.package$.error(package.scala:27)
      	at org.apache.spark.sql.catalyst.SqlParser$.parseTableIdentifier(SqlParser.scala:58)
      	at org.apache.spark.sql.SQLContext.table(SQLContext.scala:827)
      	at org.apache.spark.sql.SQLContext.dropTempTable(SQLContext.scala:763)
      	at SparkSqlContextTempTableIdentifier$.identifierCheck(SparkSqlContextTempTableIdentifier.scala:9)
      	at SparkSqlContextTempTableIdentifier$.main(SparkSqlContextTempTableIdentifier.scala:42)
      	at SparkSqlContextTempTableIdentifier.main(SparkSqlContextTempTableIdentifier.scala)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:606)
      	at sbt.Run.invokeMain(Run.scala:67)
      	at sbt.Run.run0(Run.scala:61)
      	at sbt.Run.sbt$Run$$execute$1(Run.scala:51)
      	at sbt.Run$$anonfun$run$1.apply$mcV$sp(Run.scala:55)
      	at sbt.Run$$anonfun$run$1.apply(Run.scala:55)
      	at sbt.Run$$anonfun$run$1.apply(Run.scala:55)
      	at sbt.Logger$$anon$4.apply(Logger.scala:85)
      	at sbt.TrapExit$App.run(TrapExit.scala:248)
      	at java.lang.Thread.run(Thread.java:745)
      

      Code to reproduce this bug:
      https://github.com/grzegorz-chilkiewicz/SparkSqlContextTempTableIdentifier

        Attachments

          Activity

            People

            • Assignee:
              jayadevan.m Jayadevan M
              Reporter:
              grzegorz.chilkiewicz Grzegorz Chilkiewicz
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: