Details
-
New Feature
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
Description
There are several drawbacks in the original MOB 1.0 (Moderate Object Storage) implementation, which can limit the adoption of the MOB feature:
- MOB compactions are executed in a Master as a chore, which limits scalability because all I/O goes through a single HBase Master server.
- Yarn/Mapreduce framework is required to run MOB compactions in a scalable way, but this won’t work in a stand-alone HBase cluster.
- Two separate compactors for MOB and for regular store files and their interactions can result in a data loss (see
HBASE-22075)
The design goals for MOB 2.0 were to provide 100% MOB 1.0 - compatible implementation, which is free of the above drawbacks and can be used as a drop in replacement in existing MOB deployments. So, these are design goals of a MOB 2.0:
- Make MOB compactions scalable without relying on Yarn/Mapreduce framework
- Provide unified compactor for both MOB and regular store files
- Make it more robust especially w.r.t. to data losses.
- Simplify and reduce the overall MOB code.
- Provide 100% compatible implementation with MOB 1.0.
- No migration of data should be required between MOB 1.0 and MOB 2.0 - just software upgrade.
Attachments
Attachments
Issue Links
- causes
-
HBASE-25166 MobFileCompactionChore is closing the master's shared cluster connection
- Resolved
- is related to
-
HBASE-22075 Potential data loss when MOB compaction fails
- Resolved
-
HBASE-11339 HBase MOB
- Closed
- relates to
-
HBASE-23571 Handle CompactType.MOB correctly
- Open
-
HBASE-23363 MobCompactionChore takes a long time to complete once job
- Resolved
-
HBASE-23724 Change code in StoreFileInfo to use regex matcher for mob files.
- Open
- links to