Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-14176

Cannot export & import data containing no-break space (U+00A0)

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsAdd voteVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Normal
    • Resolution: Unresolved
    • None
    • Dependencies, Legacy/Tools
    • None
    • Normal

    Description

      We were unable to export and then import the same data to Cassandra - like line breaks or the no-break space (U+00A0).

      Adding v.decode() to copyutil.py fixed most characters like line breaks etc.

      Only after using the included patch, the character U+00A0 was correctly stored in Cassandra and successfully (verifiable) exported & imported & exported.

       

      diff --git a/pylib/cqlshlib/copyutil.py b/pylib/cqlshlib/copyutil.py
      index 7f97b49..883c957 100644
      --- a/pylib/cqlshlib/copyutil.py
      +++ b/pylib/cqlshlib/copyutil.py
      @@ -1871,7 +1871,7 @@ class ImportConversion(object):
                   return bytearray.fromhex(v[2:])
       
               def convert_text(v, **_):
      -            return v
      +            return v.decode('string_escape')
       
               def convert_uuid(v, **_):
                   return UUID(v)
      diff --git a/pylib/cqlshlib/formatting.py b/pylib/cqlshlib/formatting.py
      index 803ea63..79eb691 100644
      --- a/pylib/cqlshlib/formatting.py
      +++ b/pylib/cqlshlib/formatting.py
      @@ -33,7 +33,7 @@ from util import UTC
       
       is_win = platform.system() == 'Windows'
       
      -unicode_controlchars_re = re.compile(r'[\x00-\x31\x7f-\xa0]')
      +unicode_controlchars_re = re.compile(r'[\x00-\x31]')
       controlchars_re = re.compile(r'[\x00-\x31\x7f-\xff]')
      

       

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            moneytoo Marcel Dopita Assign to me
            moneytoo Marcel Dopita
            Marcel Dopita
            Paulo Motta (Deprecated)

            Dates

              Created:
              Updated:

              Slack

                Issue deployment