[HBASE-24601] Change default Hfile storage policy from HOT to NONE for HDFS

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.2.5
• Fix Version/s: 2.5.0, 3.0.0-alpha-2
• Component/s: HFile
• Hadoop Flags: Reviewed
• Assignee: dark_num
• Reporter: dark_num
• Created: 20/Jun/20 16:55
• Updated: 04/Oct/21 04:03
• Resolved: 02/Oct/21 08:46

Description

The current HBase hfile directory specifies the HOT storage strategy by default(`hbase.hstore.blockingStoreFiles` ), which is different from the default value of HDFS heterogeneous strategy now(should be empty)

It should be set to NONE like `hbase.wal.storage.policy`'s default value on HBASE-20691, and the reason for modification is the same

Otherwise, it could have a significant impact because most HDFS Managers doesn't know the default value has been changed to HOT by hbase, for example:

HDFS manger want to set hfile's dir to ONE_SSD for speed up read, they will do:

// Set Hbas Hfile dir to ONE_SSD for speed up read
// The policy that expects all subdirectories to inherit the parent directory by default (empty by default)
bin/hdfs storagepolicies -setStoragePolicy -path /home/hbase/data  -policy ONE_SSD
bin/hdfs storagepolicies -setStoragePolicy -path /home/hbase/WALs  -policy ALL_SSD

// If we do not change Hbase default setting, we will find this
bin/hdfs storagepolicies -getStoragePolicy -path /home/hbase/data/hbase/meta/xxxx/info 
return "HOT" instead of "ONE_SSD"

And here is a pic to show the problem ↑:

 

However, if Hfile uses the HOT strategy by default, then we will find that the actually stored directory has been set to HOT, then it will not inherit the ONE_SSD of the parent directory policy, and if we set NONE in hbase, this is problem can be avoided

 Unless someone sets other values manually, then we believe the operator knows what he is doing

Issue Links

• links to: GitHub Pull Request #1940