[IMPALA-11696] Incorrect warnings when creating text/sequence table with row format delimiters

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 4.1.0, Impala 4.2.0, Impala 4.1.1
• Fix Version/s: Impala 4.1.2, Impala 4.3.0
• Component/s: Frontend
• Target Version: Impala 4.1.2
• Epic Color: ghx-label-4
• Assignee: Quanlong Huang
• Reporter: Quanlong Huang
• Created: 01/Nov/22 12:53
• Updated: 28/Mar/23 22:00
• Resolved: 06/Dec/22 04:28

Description

IMPALA-9822 adds a warning when the customized row format delimiters in the CreateTable statement is ignored. However, we see the same warning when creating TEXT or SEQUENCE_FILE tables.

[localhost:21050] default> create external table my_csv (a int, b string) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' stored as textfile;
Query: create external table my_csv (a int, b string) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' stored as textfile
+-------------------------+
| summary                 |
+-------------------------+
| Table has been created. |
+-------------------------+
WARNINGS: 'ROW FORMAT DELIMITED FIELDS TERMINATED BY ','' is ignored.
Fetched 1 row(s) in 0.11s 

The following code is buggy:

    if (getRowFormat() != null) {
      String fieldDelimiter = getRowFormat().getFieldDelimiter();
      String lineDelimiter = getRowFormat().getLineDelimiter();
      String escapeChar = getRowFormat().getEscapeChar();
      if (getFileFormat() != THdfsFileFormat.TEXT
          || getFileFormat() != THdfsFileFormat.SEQUENCE_FILE) {  // Should use && instead
        if (fieldDelimiter != null) {
          analyzer.addWarning("'ROW FORMAT DELIMITED FIELDS TERMINATED BY '"
              + fieldDelimiter + "'' is ignored.");
        }
        if (lineDelimiter != null) {
          analyzer.addWarning("'ROW FORMAT DELIMITED LINES TERMINATED BY '"
              + lineDelimiter + "'' is ignored.");
        }
        if (escapeChar != null) {
          analyzer.addWarning(
              "'ROW FORMAT DELIMITED ESCAPED BY '" + escapeChar + "'' is ignored.");
        }
      }
    }

https://github.com/apache/impala/blob/8271bdd587d241cd5a61ccae7422bbb5fafcfaf7/fe/src/main/java/org/apache/impala/analysis/CreateTableStmt.java#L276

Issue Links

• is caused by: IMPALA-9822 Impala does not notify user that row format delimited fields is only logical when using STORED AS TEXTFILE
• links to: Code Review