[HBASE-2468] Improvements to prewarm META cache on clients

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.90.0
• Component/s: Client
• Labels: moved_from_0_20_5
• Hadoop Flags: Reviewed
• Assignee: Mingjie Lai
• Reporter: Todd Lipcon
• Created: 19/Apr/10 19:58
• Updated: 20/Nov/15 12:42
• Resolved: 13/Jun/10 17:22

Description

A couple different use cases cause storms of reads to META during startup. For example, a large MR job will cause each map task to hit meta since it starts with an empty cache.

A couple possible improvements have been proposed:

• MR jobs could ship a copy of META for the table in the DistributedCache
• Clients could prewarm cache by doing a large scan of all the meta for the table instead of random reads for each miss
• Each miss could fetch ahead some number of rows in META