Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-22767

Beeline doesn't parse semicolons in comments properly

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.0.0
    • Component/s: Beeline
    • Labels:
      None

      Description

      HIVE-12646 fixed the handling of semicolons in quoted strings, but leaves the problem of semicolons in comments. E.g. with beeline connected to any database...

      this works:

      select 1; select /*   */ 2; select /*   */ 3;

      this doesn't work:

      select 1; select /* ; */ 2; select /* ; */ 3;

      This has been fixed and reintroduced before (possibly multiple times). Ideally, there should be a single utility method somewhere to separate comments, strings and commands – with the proper testing in place (q files).
      However, I'm trying to make this fix back-portable, so a light touch is needed. I'm focusing on beeline for now, and only writing (very thorough) unit tests, as I cannot exclude any new q files from TestCliDriver (which would break, since it's using a different parsing method).


      P.S. excerpt of the error message:

      0: jdbc:hive2://...> select 1; select /* ; */ 2; select /* ; */ 3;
      INFO  : Compiling command(queryId=...): select 1
      INFO  : Semantic Analysis Completed (retrial = false)
      INFO  : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0, type:int, comment:null)], properties:null)
      INFO  : Completed compiling command(queryId=...); Time taken: 0.38 seconds
      INFO  : Executing command(queryId=...): select 1
      INFO  : Completed executing command(queryId=...); Time taken: 0.004 seconds
      INFO  : OK
      +------+
      | _c0  |
      +------+
      | 1    |
      +------+
      1 row selected (2.007 seconds)
      INFO  : Compiling command(queryId=...): select /*
      ERROR : FAILED: ParseException line 1:9 cannot recognize input near '<EOF>' '<EOF>' '<EOF>' in select clause
      org.apache.hadoop.hive.ql.parse.ParseException: line 1:9 cannot recognize input near '<EOF>' '<EOF>' '<EOF>' in select clause
      	at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:233)
      	at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:79)
      	at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:72)
      	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:598)
      	at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1505)
      	at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1452)
      	at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1447)
      	at org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
      	at ...
      

      Similarly, the following query also fails:

      select /* ' */ 1; select /* ' */ 2;

      I suspect line comments are also not handled properly but I cannot reproduce this in interactive beeline...

        Attachments

        1. HIVE-22767.01.patch
          17 kB
          Zoltan Matyus
        2. HIVE-22767.02.patch
          17 kB
          Zoltan Matyus
        3. HIVE-22767.03.patch
          19 kB
          Zoltan Matyus

          Activity

            People

            • Assignee:
              zmatyus Zoltan Matyus
              Reporter:
              zmatyus Zoltan Matyus
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: