Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-22749

Distributed MOB compactions

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: mob
    • Labels:
      None

      Description

      There are several drawbacks in the original MOB 1.0 (Moderate Object Storage) implementation, which can limit the adoption of the MOB feature:

      1. MOB compactions are executed in a Master as a chore, which limits scalability because all I/O goes through a single HBase Master server.
      2. Yarn/Mapreduce framework is required to run MOB compactions in a scalable way, but this won’t work in a stand-alone HBase cluster.
      3. Two separate compactors for MOB and for regular store files and their interactions can result in a data loss (see HBASE-22075)

      The design goals for MOB 2.0 were to provide 100% MOB 1.0 - compatible implementation, which is free of the above drawbacks and can be used as a drop in replacement in existing MOB deployments. So, these are design goals of a MOB 2.0:

      1. Make MOB compactions scalable without relying on Yarn/Mapreduce framework
      2. Provide unified compactor for both MOB and regular store files
      3. Make it more robust especially w.r.t. to data losses.
      4. Simplify and reduce the overall MOB code.
      5. Provide 100% compatible implementation with MOB 1.0.
      6. No migration of data should be required between MOB 1.0 and MOB 2.0 - just software upgrade.

        Attachments

        1. HBASE-22749-branch-2.2-v4.patch
          301 kB
          Vladimir Rodionov
        2. HBASE-22749-master-v1.patch
          301 kB
          Vladimir Rodionov
        3. HBASE-22749-master-v2.patch
          319 kB
          Vladimir Rodionov
        4. HBase-MOB-2.0-v1.pdf
          171 kB
          Vladimir Rodionov
        5. HBase-MOB-2.0-v2.1.pdf
          199 kB
          Vladimir Rodionov
        6. HBase-MOB-2.0-v2.2.pdf
          195 kB
          Vladimir Rodionov
        7. HBase-MOB-2.0-v2.pdf
          203 kB
          Vladimir Rodionov

          Issue Links

            Activity

              People

              • Assignee:
                vrodionov Vladimir Rodionov
                Reporter:
                vrodionov Vladimir Rodionov
              • Votes:
                0 Vote for this issue
                Watchers:
                15 Start watching this issue

                Dates

                • Created:
                  Updated: