[ORC-842] Remove newKey from StringHashTableDictionary

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.8.0
• Fix Version/s: 1.8.0
• Component/s: Java
• Assignee: David Mollitor
• Reporter: David Mollitor
• Created: 09/Jul/21 18:30
• Updated: 03/Sep/22 22:43
• Resolved: 29/Jul/21 10:03

Description

In StringHashTableDictionary, there is an internal buffer called newKey.

See if this can be removed. When calling set on newKey, there is actually a buffer copy performed:

https://github.com/apache/hadoop/blob/6ac5d8a1ed6ec3f6de7af3b8a79869830a4377ee/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/Text.java#L240-L260

So, for the method StringHashTableDictionary#add(byte[],int,int) the byte array is copied into the newKey and then it is copied into itself again in StringHashTableDictionary#add(Text). See if it can be removed, but at least remove it from the StringHashTableDictionary#add(Text) method. The input can be acted on directly since the Text object will not be modified.

Issue Links

• is blocked by: ORC-830 Do Not Copy String When Adding to StringHashTableDictionary
• links to: GitHub Pull Request #748