[SPARK-17231] Avoid building debug or trace log messages unless the respective log level is enabled

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Fix Version/s: 2.0.1, 2.1.0
• Component/s: Spark Core

Environment

Spark cluster with 8 r3.8xl EC2 worker instances

• Assignee: Michael MacFadden
• Reporter: Michael MacFadden
• Created: 24/Aug/16 21:42
• Updated: 25/Aug/16 23:30
• Resolved: 25/Aug/16 23:30

Description

While debugging the performance of a large GraphX connected components computation, I found several places in the network-common and network-shuffle code bases where trace or debug log messages are constructed even if the respective log level is disabled. Refactoring the respective code to avoid these constructions except where necessary led to a modest but measurable reduction in task time, GC time and the ratio thereof.

Issue Links

• links to: [Github] Pull Request #14798 (mallman)
• links to: [Github] Pull Request #14811 (mallman)