Derby
  1. Derby
  2. DERBY-3157

Lexical error (EOF) when comment consists of a single character

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: 10.3.1.4, 10.4.1.3
    • Fix Version/s: None
    • Component/s: SQL
    • Urgency:
      Normal
    • Issue & fix info:
      Repro attached

      Description

      The following statement fails with an SQLException. Adding whitespace or other characters at the end of the comment makes it pass. Adding more whitespace at the beginning of the comment doesn't help.

      PreparedStatement ps = c.prepareStatement("VALUES 1 – x");

      Exception in thread "main" java.sql.SQLSyntaxErrorException: Lexical error at line 1, column 14. Encountered: <EOF> after : "".
      at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)
      at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedPreparedStatement.<init>(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedPreparedStatement20.<init>(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedPreparedStatement30.<init>(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedPreparedStatement40.<init>(Unknown Source)
      at org.apache.derby.jdbc.Driver40.newEmbedPreparedStatement(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedConnection.prepareStatement(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedConnection.prepareStatement(Unknown Source)
      at comment.main(comment.java:5)
      Caused by: java.sql.SQLException: Lexical error at line 1, column 14. Encountered: <EOF> after : "".
      at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
      ... 14 more
      Caused by: ERROR 42X02: Lexical error at line 1, column 14. Encountered: <EOF> after : "".
      at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
      at org.apache.derby.impl.sql.compile.ParserImpl.parseStatement(Unknown Source)
      at org.apache.derby.impl.sql.GenericStatement.prepMinion(Unknown Source)
      at org.apache.derby.impl.sql.GenericStatement.prepare(Unknown Source)
      at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.prepareInternalStatement(Unknown Source)
      ... 8 more

      1. DERBY-3157.diff
        8 kB
        James F. Adams

        Activity

        Hide
        Dag H. Wanvik added a comment -

        This happens when the lexer pops out of one of the states LOOKFOR_DE*
        MORE productions (because there is longer a chance of finding
        --derby-properties) and there are no more characters forthcoming.

        This is an error,
        cf. https://javacc.dev.java.net/doc/tokenmanager.html:

        "Whenever the end of file <EOF> is detected, it causes the creation
        of an <EOF> token (regardless of the current state of the lexical
        analyzer). However, if an <EOF> is detected in the middle of a
        match for a regular expression, or immediately after a MORE regular
        expression has been matched, an error is reported.

        The last condition applies, so the SKIP production:

        <IT_IS_NOT_DERBYPROPERTIES_COMMENT> SKIP :

        { <SINGLE_LINE_SQLCOMMENT: (~["\n","\r"])* ("\n"|"\r"|"\r\n")?> : DEFAULT }

        which is supposed to catch this doesn't apply unless there is at least
        one more character. So, for example all of these will fail:

        – x
        – dx
        – derby-prox
        – derby-propx

        whereas these will all work:

        – xx
        – dxx
        – derby-proxx
        – derby-propxx

        The first 'x' is chewed when in a MORE state, so next is EOF.

        Show
        Dag H. Wanvik added a comment - This happens when the lexer pops out of one of the states LOOKFOR_DE* MORE productions (because there is longer a chance of finding --derby-properties) and there are no more characters forthcoming. This is an error, cf. https://javacc.dev.java.net/doc/tokenmanager.html: "Whenever the end of file <EOF> is detected, it causes the creation of an <EOF> token (regardless of the current state of the lexical analyzer). However, if an <EOF> is detected in the middle of a match for a regular expression, or immediately after a MORE regular expression has been matched, an error is reported. The last condition applies, so the SKIP production: <IT_IS_NOT_DERBYPROPERTIES_COMMENT> SKIP : { <SINGLE_LINE_SQLCOMMENT: (~["\n","\r"])* ("\n"|"\r"|"\r\n")?> : DEFAULT } which is supposed to catch this doesn't apply unless there is at least one more character. So, for example all of these will fail: – x – dx – derby-prox – derby-propx whereas these will all work: – xx – dxx – derby-proxx – derby-propxx The first 'x' is chewed when in a MORE state, so next is EOF.
        Hide
        James F. Adams added a comment -

        Attached patch proposal DERBY-3157.diff for review.

        I simplified the processing of end of line comments in order to correct certain termination conditions. A few addtional tests were added.

        Running derbyall and suites.All.

        Show
        James F. Adams added a comment - Attached patch proposal DERBY-3157 .diff for review. I simplified the processing of end of line comments in order to correct certain termination conditions. A few addtional tests were added. Running derbyall and suites.All.
        Hide
        James F. Adams added a comment -

        Removing patch available flag. An error occurs in derbyAll. I am investigating and will post a new patch when fixed.

        Show
        James F. Adams added a comment - Removing patch available flag. An error occurs in derbyAll. I am investigating and will post a new patch when fixed.
        Hide
        Kathey Marsden added a comment -

        Triaged for 10.5.2. Set normal urgency.

        Show
        Kathey Marsden added a comment - Triaged for 10.5.2. Set normal urgency.

          People

          • Assignee:
            Unassigned
            Reporter:
            Knut Anders Hatlen
          • Votes:
            1 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:

              Development