[ARROW-13965] [C++] dynamic_casts in parquet TypedColumnWriterImpl impacting performance

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 6.0.0
• Component/s: C++
• Labels: pull-request-available
• Environment: arrow 6.0.0-SNAPSHOT on both RHEL8 (gcc 8.4.1) and MacOS 11.5.2 (clang 11.0.0)
• Flags: Patch
• External issue URL: https://github.com/apache/arrow/issues/29570
• Assignee: Edward Seidl
• Reporter: Edward Seidl
• Created: 09/Sep/21 19:07
• Updated: 11/Jan/23 08:36
• Resolved: 16/Sep/21 16:57

Description

The methods WriteDictionaryPage(), CheckDictionarySizeLimit(), WriteValues(), and WriteValuesSpaced() in TypedColumnWriterImpl (cpp/src/parquet/column_writer.cc) perform dynamic_casts of the current_dict_ object to either DictEncoder or ValueEncoderType pointers.  When calling WriteBatch() with a large number of values this is ok, but when writing batches of 1 (as when using the stream api), these dynamic casts can consume a great deal of cpu.  Using gperftools against code I wrote to do a log structured merge of several parquet files, I measured the dynamic_casts taking as much as 25% of execution time.

By modifying TypedColumnWriterImpl to save downcasted observer pointers of the appropriate types, I was able to cut my execution time from 32 to 24 seconds, validating the gpertools results.  I've attached a patch to show what I did.

Issue Links

• links to: GitHub Pull Request #11131