[HDFS-13529] Fix default trash policy emptier trigger time correctly

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.1.0, 2.7.6, 3.2.0, 2.9.2, 2.8.5
• Fix Version/s: 3.3.0, 3.2.1, 3.1.3
• Component/s: namenode
• Assignee: Xiaoqiao He
• Reporter: Xiaoqiao He
• Created: 06/May/18 18:28
• Updated: 02/Oct/19 17:14
• Resolved: 01/Aug/19 21:56

Description

Trash emptier is designed to auto trigger at UTC 00:00, however I am confused all the time that it usually triggers at a few minutes even half a hour after UTC 00:00 actually in our production cluster.
The main reason is default policy emptier thread sleep more time than as expect, since it does not consider the delete operation time cost itself. especially for a large cluster, auto trash cleaner may cost dozens of minutes.
The right way is that gets current time now before calculate end time.

      long now = Time.now();
      while (true) {
        end = ceiling(now, emptierInterval);
        try {                                     // sleep for interval
          Thread.sleep(end - now);
        } catch (InterruptedException e) {
          break;                                  // exit on interrupt
        }

        try {
          now = Time.now();
          ...... // delete trash checkpoint
        } catch (Exception e) {
          LOG.warn("RuntimeException during Trash.Emptier.run(): ", e); 
        }
      }

Issue Links

• relates to: HDFS-14586 Trash missing delete the folder which near timeout checkpoint