[HBASE-28538] BackupHFileCleaner.loadHFileRefs is very expensive

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: backup&restore
• Assignee: Unassigned
• Reporter: Bryan Beaudreault
• Created: 19/Apr/24 21:31
• Updated: 05/Jun/24 19:32

Description

I noticed some odd CPU spikes on the hmasters of one of our clusters. Turns out it had been getting lots of bulkoads (30k) and processing them was expensive. The method scans hbase and then parses the paths. Surprisingly the parsing is more expensive than the reading hbase, with the vast majority of time spent in org/apache/hadoop/fs/Path.<init>.

We should see if this is possible to be optimized. Attaching profile.