[HBASE-13985] Add configuration to skip validating HFile format when bulk loading

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.98.13
• Fix Version/s: 0.98.14, 1.2.0, 1.3.0, 2.0.0
• Labels: regionserver
• Hadoop Flags: Reviewed

Release Note

A new config, hbase.loadincremental.validate.hfile , is introduced - default to true
When set to false, checking hfile format is skipped during bulkloading.

• Assignee: Victor Xu
• Reporter: Victor Xu
• Created: 29/Jun/15 03:12
• Updated: 24/Jun/22 19:35
• Resolved: 13/Aug/15 02:04

Description

When bulk loading millions of HFile into one HTable, checking HFile format is the most time-consuming phase. Maybe we could use a parallel mechanism to increase the speed, but when it comes to millions of HFiles, it may still cost dozens of minutes. So I think it's necessary to add an option for advanced user to bulkload without checking HFile format at all.
Of course, the default value of this option should be true.