[PARQUET-1491] Conditional debug logging in InternalParquetRecordReader to reduce GC

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Not A Problem
• Component/s: parquet-avro
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: Artavazd Balaian
• Created: 11/Jan/19 21:07
• Updated: 14/Jan/19 12:26
• Resolved: 14/Jan/19 12:26

Description

Currently there is no check for the log level in https://github.com/apache/parquet-mr/blob/master/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/InternalParquetRecordReader.javaL249 which causes a lot of memory allocation and performance degradation.

Link to parquet file which was used: https://drive.google.com/open?id=1xCMZrUPWvlS4KOFO8m9EmtkvDy-SiRHq

Screenshot of Java Mission Control comparison with fix and without (link to the JFR files https://drive.google.com/open?id=1blSeF-AyAhQyRYaqVsihyzy7pJCJt7U3):

Issue Links

• links to: GitHub Pull Request #591