Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-2700

ASCII NUL characters are doubled on insert into text tables

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: Impala 2.2
    • Fix Version/s: Impala 2.7.0
    • Component/s: Backend
    • Labels:

      Description

      Steps to reproduce:

      1. In impala-shell, create a table containing at least one string column and insert a row containing a single ASCII NUL character in the string column.Duplicate the table, then duplicate the result:

      CREATE TABLE escapee (
      problem string,
      other string
      ) ROW FORMAT DELIMITED
      FIELDS TERMINATED BY '\t'
      STORED AS TEXTFILE;

      INSERT INTO TABLE escapee VALUES ("\0", "foo");

      CREATE TABLE ctas1 ROW FORMAT DELIMITED
      FIELDS TERMINATED BY '\t'
      STORED AS TEXTFILE
      AS SELECT problem, other FROM escapee;

      CREATE TABLE ctas2 ROW FORMAT DELIMITED
      FIELDS TERMINATED BY '\t'
      STORED AS TEXTFILE
      AS SELECT problem, other FROM ctas1;

      2. Cat the text of the resulting tables into local files and view with a text editor that represents ASCII NUL characters visibly (such as vim):

      hdfs dfs -cat /user/hive/warehouse/escapee/_data > escapee.txt
      vim escapee.txt

      hdfs dfs -cat /user/hive/warehouse/ctas1/_data > ctas1.txt
      vim ctas1.txt

      hdfs dfs -cat /user/hive/warehouse/ctas2/_data > ctas2.txt
      vim ctas2.txt

      Expected result:
      Each file should contain a single NUL character (represented in vim as ^@) followed by a tab character and "foo".

      Actual result:
      The first file starts with two NUL characters, the second with four, and the third with eight.

        Attachments

          Activity

            People

            • Assignee:
              anujphadke Anuj Phadke
              Reporter:
              anujphadke Anuj Phadke
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: