[HBASE-26067] Change the way on how we track store file list

• Type: Umbrella
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.6.0, 3.0.0-alpha-3
• Component/s: HFile
• Hadoop Flags: Reviewed

Release Note

Introduces the StoreFileTracker interface to HBase. This is a server-side interface which abstracts how a Store (column family) knows what files should be included in that Store. Previously, HBase relied on a listing the directory a Store used for storage to determine the files which should make up that Store.

After this feature, there are two implementations of StoreFileTrackers. The first (and default) implementation is listing the Store directory. The second is a new implementation which records files which belong to a Store within each Store. Whenever the list of files that make up a Store change, this metadata file will be updated.

This feature is notable in that it better enables HBase to function on storage systems which do not provide the typical posix filesystem semantics, most importantly, those which do not implement a file rename operation which is atomic. Storage systems which do not implement atomic renames often implement a rename as a copy and delete operation which amplifies the I/O costs by 2x.

At scale, this feature should have a 2x reduction in I/O costs when using storage systems that do not provide atomic renames, most importantly in HBase compactions and memstore flushes. See the corresponding section, "Store File Tracking", in the HBase book for more information on how to use this feature.

• Assignee: Duo Zhang
• Reporter: Duo Zhang
• Created: 06/Jul/21 03:55
• Updated: 09/May/22 14:32
• Resolved: 17/Apr/22 17:33

Description

Open a separated jira to track the work since it can not be fully included in HBASE-24749.

I think this could be a landed prior to HBASE-24749, as if this works, we could have different implementations for tracking store file list.

Issue Links

• is blocked by: HBASE-26638 Cherry-pick the ReflectionUtils improvements in HBASE-21515 to branch-2
• is related to: HBASE-24749 Direct insert HFiles and Persist in-memory HFile tracking
• relates to: HBASE-26826 Backport StoreFileTracker (HBASE-26067, HBASE-26584, and others) to branch-2.5
• relates to: HBASE-26118 The HStore.commitFile and HStore.moveFileIntoPlace almost have the same logic
• relates to: HBASE-26144 The HStore.snapshot method is never called in main code
• relates to: HBASE-26584 Further improvements on StoreFileTracker
• links to: GitHub Pull Request #3593
• links to: GitHub Pull Request #4241

Sub-Tasks

• Introduce a StoreFileTracker to abstract the store file tracking logic: Resolved
• Store the store file list by a file: Resolved
• Use StoreFileTracker when splitting and merging: Resolved
• Introduce a MigrationStoreFileTracker to support migrating from different store file tracker implementations: Resolved
• Persist the StoreFileTracker configurations to TableDescriptor when creating table: Resolved
• Should find a suitable way to let users specify the store file tracker implementation: Resolved
• [Rolling Upgrading] Persist the StoreFileTracker configurations to TableDescriptor for existing tables: Resolved
• Add more checks to prevent misconfiguration on store file tracker: Resolved
• Update ref guide to mention the new store file tracker implementations: Resolved
• Cleanup the broken store files under data directory: Resolved
• Use store file tracker when snapshoting: Resolved
• Add support for specifying store file tracker when restoring or cloning snapshot: Resolved
• CreateTableProcedure fails when FileBasedStoreFileTracker is set in global config: Resolved
• Clone snapshot doesn't load reference files into FILE SFT impl: Resolved
• Refactor StoreFileTracker implementations to expose the set method: Resolved
• CreateTableProcedure still relies on temp dir and renames when creating table FS layout: Resolved