-
Type:
Bug
-
Status: Open
-
Priority:
Major
-
Resolution: Unresolved
-
Affects Version/s: 0.24.0
-
Fix Version/s: None
-
Component/s: io, native, performance
-
Labels:None
Especially in shared HBase/MR situations, management of the OS buffer cache is important. Currently, running a big MR job will evict all of HBase's hot data from cache, causing HBase performance to really suffer. However, caching of the MR input/output is rarely useful, since the datasets tend to be larger than cache and not re-read often enough that the cache is used. Having access to the native calls posix_fadvise and sync_data_range on platforms where they are supported would allow us to do a better job of managing this cache.
- incorporates
-
HDFS-2465 Add HDFS support for fadvise readahead and drop-behind
-
- Closed
-
-
MAPREDUCE-3289 Make use of fadvise in the NM's shuffle handler
-
- Closed
-
-
MAPREDUCE-4511 Add IFile readahead
-
- Closed
-