[HBASE-25566] RoundRobinTableInputFormat

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0-alpha-1, 2.5.0, 2.3.5, 2.4.2
• Component/s: mapreduce
• Hadoop Flags: Reviewed

Release Note

Adds RoundRobinTableInputFormat, a subclass of TableInputFormat, that takes the TIF#getSplits list and resorts it so as to spread the InputFormats as broadly about the cluster as possible. RRTIF works to frustrate bunching of InputSplits on RegionServers to avoid the scenario where a few RegionServers are working hard fielding many InputSplits while others idle hosting a few or none.

• Assignee: Sudhir Reddy
• Reporter: Michael Stack
• Created: 11/Feb/21 21:24
• Updated: 29/Aug/22 12:42
• Resolved: 12/Mar/21 05:23

Description

huaxiang figured we needed a round robin on the result returned out TableInputFormat to better distribute task on a production cluster at our place-of-employ.

This seems to be an old problem. See this helpful, old Flipkart blog: https://tech.flipkart.com/is-data-locality-always-out-of-the-box-in-hadoop-not-really-2ae9c95163cb

Issue Links

• is related to: HBASE-25662 Fix spotbugs warning in RoundRobinTableInputFormat
• relates to: HBASE-27343 RoundRobinTableInputFormat main should extend `AbstractHBaseTool` for proper configuration parsing
• links to: GitHub Pull Request #2947