Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-1183

New MR splitting algorithm and other new features need a way to split a key range in N chunks

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • None
    • 0.20.0
    • util
    • None

    Description

      For HBASE-1172 and other functionality coming soon, we need to be able to take a [start,stop) range and divide it into chunks.

      For example, we have 10 regions but want to run 30 maps. We need to divide each region into three key ranges for the start/stop of each scanner.

      Implementing using java.math.BigInteger

      Will also include a couple additional helpers in Bytes to make life easy.

      Attachments

        1. hbase-1183-v1.patch
          3 kB
          Jonathan Gray
        2. hbase-1183-v2.patch
          3 kB
          Jonathan Gray
        3. hbase-1183-v3.patch
          3 kB
          Jonathan Gray
        4. hbase-1183-v4.patch
          4 kB
          Jonathan Gray

        Issue Links

        blocks

        New Feature - A new feature of the product, which has yet to be developed. HBASE-1349 Add Support to skip bad records on MR jobs

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Closed
        is depended upon by

        Improvement - An improvement or enhancement to an existing feature or task. HBASE-1172 Modify TableInputFormat splitting algorithm to allow any number of mappers

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Closed
        is related to

        New Feature - A new feature of the product, which has yet to be developed. HBASE-32 [hbase] Add row count estimator

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Closed

        Improvement - An improvement or enhancement to an existing feature or task. HBASE-635 Individual TableMap tasks don't report percentage complete.

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Closed

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            streamy Jonathan Gray
            streamy Jonathan Gray
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment