[HDDS-3943] Cleanup empty container directory

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Target Version/s: 2.0.0
• Assignee: Sammi Chen
• Reporter: Sammi Chen
• Created: 09/Jul/20 08:58
• Updated: 16/Dec/23 23:43

Description

Here is the log after datanode restart.
One thing actullay I'd like to know is why there are empty directories.

2020-07-07 22:53:55,716 [Thread-5] ERROR org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader: Missing .container file for ContainerID: 37711
2020-07-07 22:53:55,716 [Thread-5] ERROR org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader: Missing .container file for ContainerID: 37430
2020-07-07 22:53:55,716 [Thread-5] ERROR org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader: Missing .container file for ContainerID: 37528
2020-07-07 22:53:55,716 [Thread-13] ERROR org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader: Missing .container file for ContainerID: 105470
2020-07-07 22:53:55,716 [Thread-13] ERROR org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader: Missing .container file for ContainerID: 94705
2020-07-07 22:53:55,716 [Thread-5] ERROR org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader: Missing .container file for ContainerID: 70861
2020-07-07 22:53:55,716 [Thread-13] ERROR org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader: Missing .container file for ContainerID: 26296
2020-07-07 22:53:55,717 [Thread-13] ERROR org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader: Missing .container file for ContainerID: 26440
2020-07-07 22:53:55,717 [Thread-13] ERROR org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader: Missing .container file for ContainerID: 13725
2020-07-07 22:53:55,717 [Thread-13] ERROR org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader: Missing .container file for ContainerID: 13639
2020-07-07 22:53:55,717 [Thread-13] ERROR org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader: Missing .container file for ContainerID: 70989
2020-07-07 22:53:55,718 [Thread-13] ERROR org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader: Missing .container file for ContainerID: 71111
2020-07-07 22:53:55,718 [Thread-13] ERROR org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader: Missing .container file for ContainerID: 71030
2020-07-07 22:53:55,718 [Thread-13] ERROR org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader: Missing .container file for ContainerID: 9801
2020-07-07 22:53:55,718 [Thread-5] ERROR org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader: Missing .container file for ContainerID: 109694
2020-07-07 22:53:55,719 [Thread-5] ERROR org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader: Missing .container file for ContainerID: 15166
2020-07-07 22:53:55,720 [Thread-5] ERROR org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader: Missing .container file for ContainerID: 79205
2020-07-07 22:53:55,720 [Thread-5] ERROR org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader: Missing .container file for ContainerID: 78877
2020-07-07 22:53:55,721 [Thread-5] ERROR org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader: Missing .container file for ContainerID: 102541
2020-07-07 22:53:55,722 [Thread-5] ERROR org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader: Missing .container file for ContainerID: 94557
2020-07-07 22:53:55,723 [Thread-5] ERROR org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader: Missing .container file for ContainerID: 87951
2020-07-07 22:53:55,724 [Thread-5] ERROR org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader: Missing .container file for ContainerID: 87865
2020-07-07 22:53:55,724 [Thread-5] ERROR org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader: Missing .container file for ContainerID: 87752
2020-07-07 22:53:55,724 [Thread-5] ERROR org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader: Missing .container file for ContainerID: 87578
2020-07-07 22:53:55,724 [Thread-5] ERROR org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader: Missing .container file for ContainerID: 87935
2020-07-07 22:53:55,724 [Thread-5] ERROR org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader: Missing .container file for ContainerID: 87870
2020-07-07 22:53:55,724 [Thread-13] ERROR org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader: Missing .container file for ContainerID: 55222
2020-07-07 22:53:55,725 [Thread-5] ERROR org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader: Missing .container file for ContainerID: 70266
2020-07-07 22:53:55,725 [Thread-13] ERROR org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader: Missing .container file for ContainerID: 55148
2020-07-07 22:53:55,725 [Thread-5] ERROR org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader: Missing .container file for ContainerID: 70472
2020-07-07 22:53:55,725 [Thread-5] ERROR org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader: Missing .container file for ContainerID: 70291
2020-07-07 22:53:55,725 [Thread-5] ERROR org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader: Missing .container file for ContainerID: 32741
2020-07-07 22:53:55,726 [Thread-5] ERROR org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader: Missing .container file for ContainerID: 32490
2020-07-07 22:53:55,726 [Thread-5] ERROR org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader: Missing .container file for ContainerID: 32762
2020-07-07 22:53:55,727 [Thread-13] ERROR org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader: Missing .container file for ContainerID: 20787

Issue Links

• relates to: HDDS-6449 Failed container delete can leave artifacts on disk
• relates to: HDDS-4987 Import container should not delete container contents if container already exists