[HBASE-2782] QOS for META table access

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.89.20100621
• Fix Version/s: 0.90.0
• Component/s: regionserver
• Assignee: ryan rawson
• Reporter: Todd Lipcon
• Created: 24/Jun/10 01:37
• Updated: 20/Nov/15 12:43
• Resolved: 01/Oct/10 22:19

Description

I'd like to brainstorm some ideas on how we can prioritize reads and writes to META above reads and writes to other tables. I've noticed that if the regionserver hosting META is under heavy load, then lots of other operations take much longer than they should. For example, I'm currently running 120 threads of YCSB across 3 client nodes hitting a 5-node cluster. Doing a full scan of META (only 600 rows) takes upwards of 30 seconds in the shell, since all of the handler threads are tied up and there's a long RPC queue.

Issue Links

• is related to: HBASE-2964 Deadlock when RS tries to RPC to itself inside SplitTransaction
• is related to: HADOOP-6952 Support sending priority RPC
• is related to: HDFS-599 Improve Namenode robustness by prioritizing datanode heartbeats over client requests
• is related to: HBASE-2981 Improve consistency of performance during heavy load