[IMPALA-10349] Revisit constant folding on non-ASCII strings

• Type: Improvement
• Status: Open
• Priority: Critical
• Resolution: Unresolved
• Component/s: Frontend
• Epic Link: UTF-8
• Epic Color: ghx-label-9
• Assignee: Unassigned
• Reporter: Quanlong Huang
• Created: 23/Nov/20 09:31
• Updated: 10/Apr/24 17:05

Description

Constant folding may produce non-ASCII strings. In such cases, we currently abandon folding the constant. See commit message of IMPALA-1788 or codes here: https://github.com/apache/impala/blob/9672d945963e1ca3c8699340f92d7d6ce1d91c9f/fe/src/main/java/org/apache/impala/analysis/LiteralExpr.java#L274-L282

I think we should allow folding non-ASCII strings if they are legal UTF-8 strings.

Example of constant folding work:

Query: explain select * from functional.alltypes where string_col = substr('123', 1, 1)
+-------------------------------------------------------------+
| Explain String                                              |
+-------------------------------------------------------------+
| Max Per-Host Resource Reservation: Memory=32.00KB Threads=3 |
| Per-Host Resource Estimates: Memory=160MB                   |
| Codegen disabled by planner                                 |
|                                                             |
| PLAN-ROOT SINK                                              |
| |                                                           |
| 01:EXCHANGE [UNPARTITIONED]                                 |
| |                                                           |
| 00:SCAN HDFS [functional.alltypes]                          |
|    HDFS partitions=24/24 files=24 size=478.45KB             |
|    predicates: string_col = '1'                             |
|    row-size=89B cardinality=730                             |
+-------------------------------------------------------------+

Example of constant folding doesn't work:

Query: explain select * from functional.alltypes where string_col = substr('引擎', 1, 3)
+-------------------------------------------------------------+
| Explain String                                              |
+-------------------------------------------------------------+
| Max Per-Host Resource Reservation: Memory=32.00KB Threads=3 |
| Per-Host Resource Estimates: Memory=160MB                   |
| Codegen disabled by planner                                 |
|                                                             |
| PLAN-ROOT SINK                                              |
| |                                                           |
| 01:EXCHANGE [UNPARTITIONED]                                 |
| |                                                           |
| 00:SCAN HDFS [functional.alltypes]                          |
|    HDFS partitions=24/24 files=24 size=478.45KB             |
|    predicates: string_col = substr('引擎', 1, 3)            |
|    row-size=89B cardinality=730                             |
+-------------------------------------------------------------+

Issue Links

• is related to: IMPALA-5323 Support Kudu BINARY