Details
Major Description
INTERVAL is a Non-Reserved keyword in spark. "Non-Reserved keywords" have a special meaning in particular contexts and can be used as identifiers in other contexts. So by design, interval can be used as a column name.
scala> spark.sql("select interval from mytable") org.apache.spark.sql.catalyst.parser.ParseException: at least one time unit should be given for interval literal(line 1, pos 7)== SQL == select interval from mytable -------^^^ at org.apache.spark.sql.errors.QueryParsingErrors$.invalidIntervalLiteralError(QueryParsingErrors.scala:196) at org.apache.spark.sql.catalyst.parser.AstBuilder.$anonfun$parseIntervalLiteral$1(AstBuilder.scala:2481) at org.apache.spark.sql.catalyst.parser.ParserUtils$.withOrigin(ParserUtils.scala:133) at org.apache.spark.sql.catalyst.parser.AstBuilder.parseIntervalLiteral(AstBuilder.scala:2466) at org.apache.spark.sql.catalyst.parser.AstBuilder.$anonfun$visitInterval$1(AstBuilder.scala:2432) at org.apache.spark.sql.catalyst.parser.ParserUtils$.withOrigin(ParserUtils.scala:133) at org.apache.spark.sql.catalyst.parser.AstBuilder.visitInterval(AstBuilder.scala:2431) at org.apache.spark.sql.catalyst.parser.AstBuilder.visitInterval(AstBuilder.scala:57) at org.apache.spark.sql.catalyst.parser.SqlBaseParser$IntervalContext.accept(SqlBaseParser.java:17308) at org.apache.spark.sql.catalyst.parser.AstBuilder.visitChildren(AstBuilder.scala:71) at org.apache.spark.sql.catalyst.parser.SqlBaseBaseVisitor.visitIntervalLiteral(SqlBaseBaseVisitor.java:1581) at org.apache.spark.sql.catalyst.parser.SqlBaseParser$IntervalLiteralContext.accept(SqlBaseParser.java:16929) at org.apache.spark.sql.catalyst.parser.AstBuilder.visitChildren(AstBuilder.scala:71) at org.apache.spark.sql.catalyst.parser.SqlBaseBaseVisitor.visitConstantDefault(SqlBaseBaseVisitor.java:1511) at org.apache.spark.sql.catalyst.parser.SqlBaseParser$ConstantDefaultContext.accept(SqlBaseParser.java:15905) at org.apache.spark.sql.catalyst.parser.AstBuilder.visitChildren(AstBuilder.scala:71) at org.apache.spark.sql.catalyst.parser.SqlBaseBaseVisitor.visitValueExpressionDefault(SqlBaseBaseVisitor.java:1392) at org.apache.spark.sql.catalyst.parser.SqlBaseParser$ValueExpressionDefaultContext.accept(SqlBaseParser.java:15298) at org.apache.spark.sql.catalyst.parser.AstBuilder.typedVisit(AstBuilder.scala:61) at org.apache.spark.sql.catalyst.parser.AstBuilder.expression(AstBuilder.scala:1412) at org.apache.spark.sql.catalyst.parser.AstBuilder.$anonfun$visitPredicated$1(AstBuilder.scala:1548) at org.apache.spark.sql.catalyst.parser.ParserUtils$.withOrigin(ParserUtils.scala:133) at org.apache.spark.sql.catalyst.parser.AstBuilder.visitPredicated(AstBuilder.scala:1547) at org.apache.spark.sql.catalyst.parser.AstBuilder.visitPredicated(AstBuilder.scala:57) at org.apache.spark.sql.catalyst.parser.SqlBaseParser$PredicatedContext.accept(SqlBaseParser.java:14745) at org.apache.spark.sql.catalyst.parser.AstBuilder.visitChildren(AstBuilder.scala:71) at org.apache.spark.sql.catalyst.parser.SqlBaseBaseVisitor.visitExpression(SqlBaseBaseVisitor.java:1343) at org.apache.spark.sql.catalyst.parser.SqlBaseParser$ExpressionContext.accept(SqlBaseParser.java:14606) at org.apache.spark.sql.catalyst.parser.AstBuilder.typedVisit(AstBuilder.scala:61) at org.apache.spark.sql.catalyst.parser.AstBuilder.expression(AstBuilder.scala:1412) at org.apache.spark.sql.catalyst.parser.AstBuilder.$anonfun$visitNamedExpression$1(AstBuilder.scala:1434) at org.apache.spark.sql.catalyst.parser.ParserUtils$.withOrigin(ParserUtils.scala:133) at org.apache.spark.sql.catalyst.parser.AstBuilder.visitNamedExpression(AstBuilder.scala:1433) at org.apache.spark.sql.catalyst.parser.AstBuilder.visitNamedExpression(AstBuilder.scala:57) at org.apache.spark.sql.catalyst.parser.SqlBaseParser$NamedExpressionContext.accept(SqlBaseParser.java:14124) at org.apache.spark.sql.catalyst.parser.AstBuilder.typedVisit(AstBuilder.scala:61) at org.apache.spark.sql.catalyst.parser.AstBuilder.$anonfun$visitNamedExpressionSeq$2(AstBuilder.scala:628) at scala.collection.immutable.List.map(List.scala:293) at org.apache.spark.sql.catalyst.parser.AstBuilder.visitNamedExpressionSeq(AstBuilder.scala:628) at org.apache.spark.sql.catalyst.parser.AstBuilder.$anonfun$withSelectQuerySpecification$1(AstBuilder.scala:734) at org.apache.spark.sql.catalyst.parser.ParserUtils$.withOrigin(ParserUtils.scala:133) at org.apache.spark.sql.catalyst.parser.AstBuilder.withSelectQuerySpecification(AstBuilder.scala:728) at org.apache.spark.sql.catalyst.parser.AstBuilder.$anonfun$visitRegularQuerySpecification$1(AstBuilder.scala:620) at org.apache.spark.sql.catalyst.parser.ParserUtils$.withOrigin(ParserUtils.scala:133) at org.apache.spark.sql.catalyst.parser.AstBuilder.visitRegularQuerySpecification(AstBuilder.scala:608) at org.apache.spark.sql.catalyst.parser.AstBuilder.visitRegularQuerySpecification(AstBuilder.scala:57) at org.apache.spark.sql.catalyst.parser.SqlBaseParser$RegularQuerySpecificationContext.accept(SqlBaseParser.java:9679) at org.apache.spark.sql.catalyst.parser.AstBuilder.visitChildren(AstBuilder.scala:71) at org.apache.spark.sql.catalyst.parser.SqlBaseBaseVisitor.visitQueryPrimaryDefault(SqlBaseBaseVisitor.java:846) at org.apache.spark.sql.catalyst.parser.SqlBaseParser$QueryPrimaryDefaultContext.accept(SqlBaseParser.java:9184) at org.apache.spark.sql.catalyst.parser.AstBuilder.visitChildren(AstBuilder.scala:71) at org.apache.spark.sql.catalyst.parser.SqlBaseBaseVisitor.visitQueryTermDefault(SqlBaseBaseVisitor.java:832) at org.apache.spark.sql.catalyst.parser.SqlBaseParser$QueryTermDefaultContext.accept(SqlBaseParser.java:8953) at org.apache.spark.sql.catalyst.parser.AstBuilder.typedVisit(AstBuilder.scala:61) at org.apache.spark.sql.catalyst.parser.AstBuilder.plan(AstBuilder.scala:112) at org.apache.spark.sql.catalyst.parser.AstBuilder.$anonfun$visitQuery$1(AstBuilder.scala:118) at org.apache.spark.sql.catalyst.parser.ParserUtils$.withOrigin(ParserUtils.scala:133) at org.apache.spark.sql.catalyst.parser.AstBuilder.visitQuery(AstBuilder.scala:117) at org.apache.spark.sql.catalyst.parser.AstBuilder.visitQuery(AstBuilder.scala:57) at org.apache.spark.sql.catalyst.parser.SqlBaseParser$QueryContext.accept(SqlBaseParser.java:6398) at org.apache.spark.sql.catalyst.parser.AstBuilder.visitChildren(AstBuilder.scala:71) at org.apache.spark.sql.catalyst.parser.SqlBaseBaseVisitor.visitStatementDefault(SqlBaseBaseVisitor.java:69) at org.apache.spark.sql.catalyst.parser.SqlBaseParser$StatementDefaultContext.accept(SqlBaseParser.java:1835) at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visit(AbstractParseTreeVisitor.java:18) at org.apache.spark.sql.catalyst.parser.AstBuilder.$anonfun$visitSingleStatement$1(AstBuilder.scala:78) at org.apache.spark.sql.catalyst.parser.ParserUtils$.withOrigin(ParserUtils.scala:133) at org.apache.spark.sql.catalyst.parser.AstBuilder.visitSingleStatement(AstBuilder.scala:78) at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.$anonfun$parsePlan$1(ParseDriver.scala:78) at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parse(ParseDriver.scala:110) at org.apache.spark.sql.execution.SparkSqlParser.parse(SparkSqlParser.scala:51) at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parsePlan(ParseDriver.scala:77) at org.apache.spark.sql.SparkSession.$anonfun$sql$2(SparkSession.scala:616) at org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:111) at org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:616) at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:775) at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:613) ... 47 elided
Since there must be at least one time unit after the interval, why in SqlBaseParser.g4, the definition of interval is
interval
: INTERVAL (errorCapturingMultiUnitsInterval | errorCapturingUnitToUnitInterval)?
;
instead of
interval
: INTERVAL (errorCapturingMultiUnitsInterval | errorCapturingUnitToUnitInterval)
;
If we remove the "?", we ensure that there must be at least one time unit after the interval from the parsing level
