[CASSANDRA-14176] Cannot export & import data containing no-break space (U+00A0)

• Type: Bug
• Status: Open
• Priority: Normal
• Resolution: Unresolved
• Component/s: Dependencies, Legacy/Tools
• Severity: Normal
• Assignee: Marcel Dopita
• Reporter: Marcel Dopita

Authors

Marcel Dopita

Reviewers

Paulo Motta

• Created: 19/Jan/18 09:32
• Updated: 16/Apr/19 09:29

Description

We were unable to export and then import the same data to Cassandra - like line breaks or the no-break space (U+00A0).

Adding v.decode() to copyutil.py fixed most characters like line breaks etc.

Only after using the included patch, the character U+00A0 was correctly stored in Cassandra and successfully (verifiable) exported & imported & exported.

 

diff --git a/pylib/cqlshlib/copyutil.py b/pylib/cqlshlib/copyutil.py
index 7f97b49..883c957 100644
--- a/pylib/cqlshlib/copyutil.py
+++ b/pylib/cqlshlib/copyutil.py
@@ -1871,7 +1871,7 @@ class ImportConversion(object):
             return bytearray.fromhex(v[2:])
 
         def convert_text(v, **_):
-            return v
+            return v.decode('string_escape')
 
         def convert_uuid(v, **_):
             return UUID(v)
diff --git a/pylib/cqlshlib/formatting.py b/pylib/cqlshlib/formatting.py
index 803ea63..79eb691 100644
--- a/pylib/cqlshlib/formatting.py
+++ b/pylib/cqlshlib/formatting.py
@@ -33,7 +33,7 @@ from util import UTC
 
 is_win = platform.system() == 'Windows'
 
-unicode_controlchars_re = re.compile(r'[\x00-\x31\x7f-\xa0]')
+unicode_controlchars_re = re.compile(r'[\x00-\x31]')
 controlchars_re = re.compile(r'[\x00-\x31\x7f-\xff]')