[HIVE-18217] When Text is corrupted, populateMappings() hangs indefinitely

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.3.2
• Assignee: Unassigned
• Reporter: John Doe
• Created: 05/Dec/17 02:19
• Updated: 05/Dec/17 02:19

Description

Similar like HIVE-18216,
when the Text is corrupted, the following loop become infinite.

  private void populateMappings(Text from, Text to) {
    replacementMap.clear();
    deletionSet.clear();

    ByteBuffer fromBytes = ByteBuffer.wrap(from.getBytes(), 0, from.getLength());
    ByteBuffer toBytes = ByteBuffer.wrap(to.getBytes(), 0, to.getLength());

    // Traverse through the from string, one code point at a time
    while (fromBytes.hasRemaining()) {
      // This will also move the iterator ahead by one code point
      int fromCodePoint = Text.bytesToCodePoint(fromBytes);
      // If the to string has more code points, make sure to traverse it too
      if (toBytes.hasRemaining()) {
        int toCodePoint = Text.bytesToCodePoint(toBytes);
        // If the code point from from string already has a replacement or is to be deleted, we
        // don't need to do anything, just move on to the next code point
        if (replacementMap.containsKey(fromCodePoint) || deletionSet.contains(fromCodePoint)) {
          continue;
        }
        replacementMap.put(fromCodePoint, toCodePoint);
      } else {
        // If the code point from from string already has a replacement or is to be deleted, we
        // don't need to do anything, just move on to the next code point
        if (replacementMap.containsKey(fromCodePoint) || deletionSet.contains(fromCodePoint)) {
          continue;
        }
        deletionSet.add(fromCodePoint);
      }
    }
  }