[HBASE-20554] "WALs outstanding" message from CleanerChore is noisy

• Type: Bug
• Status: Resolved
• Priority: Trivial
• Resolution: Fixed
• Fix Version/s: 3.0.0-alpha-1, 2.1.0, 1.3.3, 2.0.1, 1.4.5
• Assignee: Andrew Kyle Purtell
• Reporter: Andrew Kyle Purtell
• Created: 09/May/18 16:53
• Updated: 01/Feb/19 20:05
• Resolved: 10/May/18 02:15

Description

WARN level "WALs outstanding" from CleanerChore should be DEBUG and are not always correct.

I left a cluster configured for ITBLL (retaining all WALs for post hoc analysis) and in the morning found the master log full of "WALs outstanding" warnings from CleanerChore.

Should this really be a warning?

2018-05-09 16:42:03,893 WARN [node-1.cluster,16000,1525851521469_ChoreService_2] cleaner.CleanerChore: WALs outstanding under hdfs://node-1.cluster/hbase/oldWALs

If someone has configured really long WAL retention then having WALs in oldWALs will be normal.

Also, it seems the warning is sometimes incorrect.

2018-05-09 16:42:24,751 WARN [node-1.cluster,16000,1525851521469_ChoreService_1] cleaner.CleanerChore: WALs outstanding under hdfs://node-1.cluster/hbase/archive

There are no WALs under archive/.

Even at DEBUG level, if it is not correct, then it can lead an operator to be concerned about nothing, so better to just remove it.