Details
-
New Feature
-
Status: Open
-
Major
-
Resolution: Unresolved
-
3.0.0-alpha1
-
None
-
None
Description
Currently the datanode read/write SLA is difficult to be guaranteed for HBase online requirement. One of major reasons is we don't support io priority or io request reorder inside datanode.
I propose introducing a self-maintain io queue mechanism to handle io request priority. Imaging there're lots of concurrent read/write requests from HBase side, and a background datanode block scanner is running(default is every 21 days, IIRC) just in time, then the HBase read/write 99% or 99.9% percentile latency would be vulnerable despite we have a bg thread throttling...
the reorder stuff i have not thought clearly enough, but definitely the reorder in the queue in the app side would beat the currently relying OS's io queue merge.