While looking at log files for SMB joins in hive, it was noticed that the actual join op didn't show up as a significant fraction of the time spent. Most of the time was spent parsing configuration files.
To confirm, I put log lines in the HiveConf constructor and eventually made the following edit to the code
And re-ran my query to compare timings.
|Cumulative CPU||731.07 sec||386.0 sec|
|Total time||347.66 seconds||218.855 seconds|
The query used was
On a scale=2 tpcds data-set, where both store_sales & inventory are bucketed into 4 buckets, with store_sales split into 7 partitions and inventory into 261 partitions.
78% of all CPU time was spent within new HiveConf(). The yourkit profiler runs are attached.