[IMPALA-10299] Impala-shell hangs in printing partial UTF-8 characters

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 4.0.0
• Fix Version/s: Impala 4.0.0
• Component/s: Clients
• Epic Link: UTF-8
• Epic Color: ghx-label-5
• Assignee: Quanlong Huang
• Reporter: Quanlong Huang
• Created: 28/Oct/20 03:58
• Updated: 23/May/22 16:37
• Resolved: 10/Nov/20 04:57

Description

Saw the hanging when playing around with Chinese characters:

[localhost:21050] default> select substr("引擎", 1, 3);
Query: select substr("引擎", 1, 3)
Query submitted at: 2020-10-28 11:54:26 (Coordinator: http://quanlong-OptiPlex-BJ:25000)
Query progress can be monitored at: http://quanlong-OptiPlex-BJ:25000/query_plan?query_id=464f44d405bb16a1:5b41e00e00000000
+----------------------+
| substr('引擎', 1, 3) |
+----------------------+
| 引                   |
+----------------------+
Fetched 1 row(s) in 0.13s
[localhost:21050] default> select substr("引擎", 1, 2);
Query: select substr("引擎", 1, 2)
Query submitted at: 2020-10-28 11:54:32 (Coordinator: http://quanlong-OptiPlex-BJ:25000)
Query progress can be monitored at: http://quanlong-OptiPlex-BJ:25000/query_plan?query_id=d048231be08f29b4:860c959500000000
Caught exception 'utf8' codec can't decode bytes in position 0-1: unexpected end of data, type=<type 'exceptions.UnicodeDecodeError'> in FetchResults.

This seems not a problem in previous versions. E.g. see the examples in the description of IMPALA-9662.

Issue Links

• is caused by: THRIFT-5303 Unicode decode errors in _fast_decode
• relates to: IMPALA-11313 impala-shell's PyPi form factor still suffers from IMPALA-10299
• relates to: IMPALA-10145 UnicodeDecodeError in Thrift 0.11.0 generated files