[HIVE-4758] NULLs and record separators broken with vectorization branch intermediate outputs

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: vectorization-branch
• Fix Version/s: vectorization-branch, 0.13.0
• Component/s: Query Processor
• Release Note: Fix the NULL serialization and record separator insertion for VectorizedRowBatch.toString()
• Assignee: Gopal Vijayaraghavan
• Reporter: Gopal Vijayaraghavan
• Created: 19/Jun/13 16:22
• Updated: 23/Oct/13 21:59
• Resolved: 19/Jun/13 22:06

Description

Queries of type timestamp on partitioned tables return NULL for all rows of timestamp columns, if the first row in the column is NULL.

This was tracked down to the failure of timestamp columns to parse the map output properly, which was due to differing format from the unvectorized code's output.

The output file for vectorized code says

(null)^A
2013-02-12 21:05:29^A

Where the unvectorized code outputs

\N
2013-02-12 21:05:29

The vectorized code passes on the "(null)" string to the LazyTimestamp parser, which fails to parse it & returns "NULL", but slowed down massively by the IllegalArgumentException.

And the extraneous ^A prevents the actual Timestamp from being parsed into valid timestamps.