[ORC-386] Add new spark file format benchmark

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.6.0
• Assignee: Owen O'Malley
• Reporter: Owen O'Malley
• Created: 06/Jul/18 21:20
• Updated: 04/Sep/19 02:00
• Resolved: 13/Jul/18 21:34

Description

I've done a new set of benchmarks that compare the performance of the file formats as they are read from Spark.

As part of this, I've refactored a lot of the benchmark code into different modules so that we don't need to fight quite as many dependency hell fires when trying to build an uber jar with both Spark and Hive in it.

Issue Links

• is related to: ORC-454 Use Spark 2.4.0 in benchmark
• links to: GitHub Pull Request #290