[ORC-406] ORC: Char(n) and Varchar(n) writers truncate to n bytes & corrupts multi-byte data

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.5.2
• Fix Version/s: 1.5.3, 1.6.0
• Assignee: Gopal Vijayaraghavan
• Reporter: Gopal Vijayaraghavan
• Created: 18/Sep/18 19:04
• Updated: 09/Oct/18 07:04
• Resolved: 20/Sep/18 23:01

Description

https://github.com/apache/orc/blob/master/java/core/src/java/org/apache/orc/impl/writer/CharTreeWriter.java#L41

    itemLength = schema.getMaxLength();
    padding = new byte[itemLength];
  }

https://github.com/apache/orc/blob/master/java/core/src/java/org/apache/orc/impl/writer/VarcharTreeWriter.java#L48

      if (vector.noNulls || !vector.isNull[0]) {
        int itemLength = Math.min(vec.length[0], maxLength);

Issue Links

• is cloned by: ORC-412 [C++] ORC: Char(n) and Varchar(n) writers truncate to n bytes & corrupts multi-byte data
• is related to: HIVE-20520 length(CHAR) doesn't consider trailing space
• links to: GitHub Pull Request #310