[HIVE-3168] LazyBinaryObjectInspector.getPrimitiveJavaObject copies beyond length of underlying BytesWritable

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.9.0
• Fix Version/s: 0.10.0
• Component/s: Serializers/Deserializers
• Assignee: Thejas Nair
• Reporter: Thejas Nair
• Created: 21/Jun/12 02:53
• Updated: 10/Jan/13 19:53
• Resolved: 12/Jul/12 16:53

Description

LazyBinaryObjectInspector.getPrimitiveJavaObject copies the full capacity of the LazyBinary's underlying BytesWritable object, which can be greater than the size of the actual contents.

This leads to additional characters at the end of the ByteArrayRef returned. When the LazyBinary object gets re-used, there can be remnants of the later portion of previous entry.
This was not seen while reading through hive queries, which I think is because a copy elsewhere seems to create LazyBinary with length == capacity. (probably LazyBinary copy constructor). This was seen when MR or pig used Hcatalog to read the data.

Issue Links

• is duplicated by: HCATALOG-430 pig/MR jobs get incorrect binary type data from hcat table