[HBASE-19478] Utilize multi-get to speed up WAL file checking in BackupLogCleaner

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0-alpha-1
• Hadoop Flags: Reviewed
• Assignee: Toshihiro Suzuki
• Reporter: Ted Yu
• Created: 10/Dec/17 20:10
• Updated: 01/Jan/18 20:45
• Resolved: 01/Jan/18 17:55

Description

Currently BackupLogCleaner#getDeletableFiles() issues one Get per WAL file:

      for (FileStatus file : files) {
        String wal = file.getPath().toString();
        boolean logInSystemTable = table.isWALFileDeletable(wal);

This is rather inefficient considering the number of WAL files in production can get quite large.

We should use multi-get to reduce the number of calls to backup table (which normally resides on another server).