[SPARK-42686] TaskMemoryManager debug logging is expensive

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.2
• Fix Version/s: 3.5.0
• Component/s: Spark Core
• Assignee: Alkis Evlogimenos
• Reporter: Alkis Evlogimenos
• Created: 06/Mar/23 16:11
• Updated: 10/Mar/23 01:13
• Resolved: 10/Mar/23 01:13

Description

TaskMemoryManager debug logging is expensive mostly because formatting operations are done eagerly and some of them are quite expensive (i.e. humanized strings). This causes visible CPU usage in scan benchmarks for example.

Issue Links

• links to: [Github] Pull Request #40302 (alkis)