Uploaded image for project: 'Phoenix'
  1. Phoenix
  2. PHOENIX-3246

U+2002 (En Space) not handled as whitespace in grammar

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 4.9.0, 4.8.1
    • None
    • None

    Description

      I had the goofiest query issue the other day. A seemingly fine query was throwing a parse error via sqlline.

      Error: ERROR 601 (42P00): Syntax error. Unexpected char: ' ' (state=42P00,code=601)
      org.apache.phoenix.exception.PhoenixParserException: ERROR 601 (42P00): Syntax error. Unexpected char: ' '
             	at org.apache.phoenix.exception.PhoenixParserException.newException(PhoenixParserException.java:33)
             	at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:118)
             	at org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1280)
             	at org.apache.phoenix.jdbc.PhoenixStatement.parseStatement(PhoenixStatement.java:1363)
             	at org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1434)
             	at sqlline.Commands.execute(Commands.java:822)
             	at sqlline.Commands.sql(Commands.java:732)
             	at sqlline.SqlLine.dispatch(SqlLine.java:807)
             	at sqlline.SqlLine.runCommands(SqlLine.java:1710)
             	at sqlline.Commands.run(Commands.java:1285)
             	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
             	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
             	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
             	at java.lang.reflect.Method.invoke(Method.java:606)
             	at sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
             	at sqlline.SqlLine.dispatch(SqlLine.java:803)
             	at sqlline.SqlLine.initArgs(SqlLine.java:613)
             	at sqlline.SqlLine.begin(SqlLine.java:656)
             	at sqlline.SqlLine.start(SqlLine.java:398)
             	at sqlline.SqlLine.main(SqlLine.java:292)
      Caused by: java.lang.RuntimeException: Unexpected char: ' '
             	at org.apache.phoenix.parse.PhoenixSQLLexer.mOTHER(PhoenixSQLLexer.java:4324)
             	at org.apache.phoenix.parse.PhoenixSQLLexer.mTokens(PhoenixSQLLexer.java:5437)
             	at org.apache.phoenix.shaded.org.antlr.runtime.Lexer.nextToken(Lexer.java:85)
             	at org.apache.phoenix.shaded.org.antlr.runtime.BufferedTokenStream.fetch(BufferedTokenStream.java:143)
             	at org.apache.phoenix.shaded.org.antlr.runtime.BufferedTokenStream.sync(BufferedTokenStream.java:137)
             	at org.apache.phoenix.shaded.org.antlr.runtime.CommonTokenStream.skipOffTokenChannels(CommonTokenStream.java:113)
             	at org.apache.phoenix.shaded.org.antlr.runtime.CommonTokenStream.LT(CommonTokenStream.java:102)
             	at org.apache.phoenix.shaded.org.antlr.runtime.BufferedTokenStream.LA(BufferedTokenStream.java:174)
             	at org.apache.phoenix.shaded.org.antlr.runtime.BaseRecognizer.mismatchIsUnwantedToken(BaseRecognizer.java:127)
             	at org.apache.phoenix.parse.PhoenixSQLParser.recoverFromMismatchedToken(PhoenixSQLParser.java:354)
             	at org.apache.phoenix.shaded.org.antlr.runtime.BaseRecognizer.match(BaseRecognizer.java:115)
             	at org.apache.phoenix.parse.PhoenixSQLParser.parseNoReserved(PhoenixSQLParser.java:9969)
             	at org.apache.phoenix.parse.PhoenixSQLParser.identifier(PhoenixSQLParser.java:9936)
             	at org.apache.phoenix.parse.PhoenixSQLParser.column_def(PhoenixSQLParser.java:3938)
             	at org.apache.phoenix.parse.PhoenixSQLParser.column_defs(PhoenixSQLParser.java:3858)
             	at org.apache.phoenix.parse.PhoenixSQLParser.create_table_node(PhoenixSQLParser.java:1104)
             	at org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:816)
             	at org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:508)
             	at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108)
             	... 18 more
      

      Re-typing the statement by hand worked successfully.

      After some hexdump and diff action, I finally found out that some of the space characters in the statement were not the normal ASCII 0x20 character, but actually the unicode U+2002 "En Space" character. They look identical to the eye which spawned all of the confusion.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            elserj Josh Elser
            elserj Josh Elser
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment