Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-12982

SQLContext: temporary table registration does not accept valid identifier

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotVotersStop watchingWatchersCreate sub-taskConvert to sub-taskLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Resolved
    • 1.6.0
    • 2.0.0
    • SQL

    Description

      We have encountered very strange behavior of SparkSQL temporary table registration.
      What identifiers for temporary table should be valid?
      Alphanumerical + '_' with at least one non-digit?

      Valid identifiers:
      df
      674123a
      674123_
      a0e97c59_4445_479d_a7ef_d770e3874123
      1ae97c59_4445_479d_a7ef_d770e3874123
      Invalid identifier:
      10e97c59_4445_479d_a7ef_d770e3874123

      Stack trace:

      java.lang.RuntimeException: [1.1] failure: identifier expected
      
      10e97c59_4445_479d_a7ef_d770e3874123
      ^
      	at scala.sys.package$.error(package.scala:27)
      	at org.apache.spark.sql.catalyst.SqlParser$.parseTableIdentifier(SqlParser.scala:58)
      	at org.apache.spark.sql.SQLContext.table(SQLContext.scala:827)
      	at org.apache.spark.sql.SQLContext.dropTempTable(SQLContext.scala:763)
      	at SparkSqlContextTempTableIdentifier$.identifierCheck(SparkSqlContextTempTableIdentifier.scala:9)
      	at SparkSqlContextTempTableIdentifier$.main(SparkSqlContextTempTableIdentifier.scala:42)
      	at SparkSqlContextTempTableIdentifier.main(SparkSqlContextTempTableIdentifier.scala)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:606)
      	at sbt.Run.invokeMain(Run.scala:67)
      	at sbt.Run.run0(Run.scala:61)
      	at sbt.Run.sbt$Run$$execute$1(Run.scala:51)
      	at sbt.Run$$anonfun$run$1.apply$mcV$sp(Run.scala:55)
      	at sbt.Run$$anonfun$run$1.apply(Run.scala:55)
      	at sbt.Run$$anonfun$run$1.apply(Run.scala:55)
      	at sbt.Logger$$anon$4.apply(Logger.scala:85)
      	at sbt.TrapExit$App.run(TrapExit.scala:248)
      	at java.lang.Thread.run(Thread.java:745)
      

      Code to reproduce this bug:
      https://github.com/grzegorz-chilkiewicz/SparkSqlContextTempTableIdentifier

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            jayadevan.m Jayadevan M Assign to me
            grzegorz.chilkiewicz Grzegorz Chilkiewicz
            Votes:
            0 Vote for this issue
            Watchers:
            7 Stop watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Issue deployment