[CASSANDRA-16429] Fix incorrect encoding for strings can be UTF8

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 4.0-rc1, 4.0
• Component/s: CQL/Interpreter
• Bug Category: Correctness
• Severity: Normal
• Complexity: Normal
• Discovered By: User Report
• Platform: All
• Since Version: 4.0-alpha1

Source Control Link

https://github.com/apache/cassandra/commit/f32475a839e01e4eea3989871d293d70e8a360d7

Test and Documentation Plan

unit, circle

• Assignee: Yifan Cai
• Reporter: Yoshi Kimoto

Authors

Yifan Cai

Reviewers

Ekaterina Dimitrova

• Created: 06/Feb/21 13:00
• Updated: 16/Mar/22 15:30
• Resolved: 23/Feb/21 03:00

Description

Tables created with Japanese character name columns are working well in C* 3.11.10 when doing a SELECT * in cqlsh but will show as garbled (shown as "?") in 4.0-beta4. DESCRIBE shows the column names correctly in both cases.

Run the attached jptest.cql script in both envs with cqlsh -f. They will yield different results.

My test env (MacOS 10.15.7):

C* 3.11.10 with

• OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_252-b09)
• Python 2.7.16

C* 4.0-beta4

• OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.9.1+1)
• Python 3.8.2

Issue Links

• is caused by: CASSANDRA-15410 Avoid over-allocation of bytes for UTF8 string serialization