Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-25566

RoundRobinTableInputFormat

    XMLWordPrintableJSON

    Details

    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Adds RoundRobinTableInputFormat, a subclass of TableInputFormat, that takes the TIF#getSplits list and resorts it so as to spread the InputFormats as broadly about the cluster as possible. RRTIF works to frustrate bunching of InputSplits on RegionServers to avoid the scenario where a few RegionServers are working hard fielding many InputSplits while others idle hosting a few or none.
      Show
      Adds RoundRobinTableInputFormat, a subclass of TableInputFormat, that takes the TIF#getSplits list and resorts it so as to spread the InputFormats as broadly about the cluster as possible. RRTIF works to frustrate bunching of InputSplits on RegionServers to avoid the scenario where a few RegionServers are working hard fielding many InputSplits while others idle hosting a few or none.

      Description

      Hua Xiang figured we needed a round robin on the result returned out TableInputFormat to better distribute task on a production cluster at our place-of-employ.

      This seems to be an old problem. See this helpful, old Flipkart blog: https://tech.flipkart.com/is-data-locality-always-out-of-the-box-in-hadoop-not-really-2ae9c95163cb

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                discoversudhir Sudhir Reddy
                Reporter:
                stack Michael Stack
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: