[HIVE-11131] Get row information on DataWritableWriter once for better writing performance

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.2.0
• Fix Version/s: 2.0.0
• Assignee: Sergio Peña
• Reporter: Sergio Peña
• Created: 26/Jun/15 22:20
• Updated: 16/Feb/16 23:51
• Resolved: 09/Jul/15 21:38

Description

DataWritableWriter is a class used to write Hive records to Parquet files. This class is getting all the information about how to parse a record, such as schema and object inspector, every time a record is written (or write() is called).

We can make this class perform better by initializing some writers per data
type once, and saving all object inspectors on each writer.

The class expects that the next records written will have the same object inspectors and schema, so there is no need to have conditions for that. When a new schema is written, DataWritableWriter is created again by Parquet.

Issue Links

• links to: Review Board