[HDFS-5366] recaching improvements

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0-alpha1
• Fix Version/s: 2.3.0
• Component/s: namenode
• Assignee: Colin McCabe
• Reporter: Colin McCabe
• Created: 16/Oct/13 01:46
• Updated: 12/May/16 18:14
• Resolved: 15/Nov/13 19:10

Description

There are a few things about our HDFS-4949 recaching strategy that could be improved.

• We should monitor the DN's maximum and current mlock'ed memory consumption levels, so that we don't ask the DN to do stuff it can't.

• We should not try to initiate caching on stale or decomissioning DataNodes (although we should not recache things stored on such nodes until they're declared dead).

• We might want to resend the DNA_CACHE or DNA_UNCACHE command a few times before giving up. Currently, we only send it once.

Issue Links

• is blocked by: HDFS-5394 fix race conditions in DN caching and uncaching
• is related to: HDFS-5507 Consider maximum DN memory, stale status when scheduling recaching
• relates to: HDFS-5096 Automatically cache new data added to a cached path