[SPARK-21271] UnsafeRow.hashCode assertion when sizeInBytes not multiple of 8

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.2.0
• Fix Version/s: 2.3.0
• Component/s: SQL
• Assignee: Kazuaki Ishizaki
• Reporter: Bogdan Raducanu
• Created: 30/Jun/17 18:35
• Updated: 27/Jul/17 07:28
• Resolved: 27/Jul/17 07:27

Description

The method is:

public int hashCode() {
    return Murmur3_x86_32.hashUnsafeWords(baseObject, baseOffset, sizeInBytes, 42);
  }

but sizeInBytes is not always a multiple of 8 (in which case hashUnsafeWords throws assertion) - for example here:

FixedLengthRowBasedKeyValueBatch.appendRow

The fix could be to use hashUnsafeBytes or to use hashUnsafeWords but on a prefix that is multiple of 8.

Issue Links

• links to: [Github] Pull Request #18503 (kiszk)