[IMPALA-4170] Incorrect double quoting of identifier in SQL generated by COMPUTE INCREMENTAL STATS.

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: Impala 2.6.0, Impala 2.7.0
• Fix Version/s: Impala 2.5.5, Impala 2.6.3, Impala 2.8.0
• Component/s: Frontend
• Labels: compute-stats regression usability
• Target Version: Impala 2.5.5, Impala 2.6.3, Impala 2.8.0
• Assignee: Alexander Behm
• Reporter: Alexander Behm
• Created: 20/Sep/16 15:00
• Updated: 15/Nov/16 13:06
• Resolved: 21/Sep/16 03:43

Description

COMPUTE INCREMENTAL STATS fails to update the stats of new partitions if the name of a partition column is a reserved word. A syntax error is reported because a generated SQL statement incorrectly uses two sets of backticks for quoting the identifier.

Repro:

create table t (i int) partitioned by (`date` int);
insert into t partition(`date`) values (1, 1);
compute incremental stats t;
insert into t partition(`date`) values (2, 2);
[localhost:21000] > compute incremental stats t;
Query: compute incremental stats t
WARNINGS: AnalysisException: Syntax error in line 1:
...te` FROM default.t WHERE (``date``=2) GROUP BY `date`
                             ^
Encountered: EMPTY IDENTIFIER
Expected: CASE, CAST, EXISTS, FALSE, IF, INTERVAL, NOT, NULL, SELECT, TRUNCATE, TRUE, VALUES, WITH, IDENTIFIER

CAUSED BY: Exception: Syntax error

The problem is in HdfsPartition.getConjunctSql() where the double quoting is pretty obvious:

  public String getConjunctSql() {
    List<String> partitionCols = Lists.newArrayList();
    for (int i = 0; i < getTable().getNumClusteringCols(); ++i) {
      partitionCols.add(ToSqlUtils.getIdentSql(getTable().getColumns().get(i).getName()));
    }

    List<String> conjuncts = Lists.newArrayList();
    for (int i = 0; i < partitionCols.size(); ++i) {
      LiteralExpr expr = getPartitionValues().get(i);
      String sql = expr.toSql();
      if (expr instanceof NullLiteral || sql.isEmpty()) {
        conjuncts.add(ToSqlUtils.getIdentSql(partitionCols.get(i))
            + " IS NULL");
      } else {
        conjuncts.add(ToSqlUtils.getIdentSql(partitionCols.get(i))
            + "=" + sql);
      }
    }
    return "(" + Joiner.on(" AND " ).join(conjuncts) + ")";
  }