[HDDS-633] HBase service fails to start with OzoneFS as defaultFS

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 0.3.0
• Component/s: Ozone Filesystem
• Target Version/s: 1.4.0
• Assignee: Unassigned
• Reporter: Soumitra Sulav
• Created: 11/Oct/18 15:59
• Updated: 18/Jul/23 17:42
• Resolved: 18/Jul/23 17:42

Description

Hbase service fails to start when hbase.root-dir is pointing to OzoneFS.

Below is the exception encountered :

2018-10-10 20:12:52,255 ERROR [master/hcatest-4:17000] master.HMaster: Failed to become active master
java.lang.IllegalStateException: The procedure WAL relies on the ability to hsync for proper operation during component failures, but the underlying filesystem does not support doing so. Please check the config value of 'hbase.procedure.store.wal.use.hsync' to set the desired level of robustness and ensure the config value of 'hbase.wal.dir' points to a FileSystem mount that can provide it.
at org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.rollWriter(WALProcedureStore.java:1047)
at org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.recoverLease(WALProcedureStore.java:383)
at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.start(ProcedureExecutor.java:551)
at org.apache.hadoop.hbase.master.HMaster.startProcedureExecutor(HMaster.java:1243)
at org.apache.hadoop.hbase.master.HMaster.startServiceThreads(HMaster.java:1162)
at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:853)
at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2040)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:553)
at java.lang.Thread.run(Thread.java:745)
2018-10-10 20:12:52,256 ERROR [master/hcatest-4:17000] master.HMaster: Master server abort: loaded coprocessors are: []
2018-10-10 20:12:52,256 ERROR [master/hcatest-4:17000] master.HMaster: ***** ABORTING master hcatest-4.openstacklocal,17000,1539202359437: Unhandled exception. Starting shutdown. *****

Above exception was also observed with YARN-atsv2 service as it uses embedded hbase and hence, WALs for storage. When I pointed the storage back to hdfs, exception was gone.

Issue Links

• duplicates: HDDS-7593 Supporting HSync and lease recovery