[HDFS-4184] Add the ability for Client to provide more hint information for DataNode to manage the OS buffer cache more accurate

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Assignee: Unassigned
• Reporter: Lijin Bin
• Created: 13/Nov/12 02:53
• Updated: 28/May/13 17:33
• Resolved: 28/May/13 17:33

Description

HDFS now has the ability to use posix_fadvise and sync_data_range syscalls to manage the OS buffer cache.

When hbase read hlog the data we can set dfs.datanode.drop.cache.behind.reads to true to drop data out of the buffer cache when performing sequential reads.
When hbase write hlog we can set dfs.datanode.drop.cache.behind.writes to true to drop data out of the buffer cache after writing
When hbase read hfile during compaction we can set dfs.datanode.readahead.bytes to a non-zero value to trigger readahead for sequential reads， and also set dfs.datanode.drop.cache.behind.reads to true to drop data out of the buffer cache when performing sequential reads.
and so on... 

Current we can only set these feature global in datanode，we should set these feature per session.

Issue Links

• duplicates: HDFS-4817 make HDFS advisory caching configurable on a per-file basis
• relates to: HDFS-2465 Add HDFS support for fadvise readahead and drop-behind