Minor Description
In LazyPersistFileScrubber.run() we have:
try {
clearCorruptLazyPersistFiles();
} catch (Exception e) {
FSNamesystem.LOG.error(
"Ignoring exception in LazyPersistFileScrubber:", e);
}
First problem is that catching Exception is sloppy. It should instead be a multicatch for the actual exceptions thrown or better a set of separate catch statements that react appropriately to the type of exception.
Second problem is that it's bad to log an ERROR that's not actionable and that can be safely ignored. The log message should be logged at WARN or INFO level.
Third, the log message is useless. If it's going to be a WARN or ERROR, a log message should be actionable. Otherwise it's an info. A log message should contain enough information for an admin to understand what it means.
In the end, I think the right thing here is to leave the high-level behavior unchanged: log a message and ignore the error, hoping that the next run will go better.