[HBASE-6339] Bulkload call to RS should begin holding write lock only after the file has been transferred

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Invalid
• Affects Version/s: 0.90.0
• Component/s: Client, regionserver
• Assignee: Unassigned
• Reporter: Harsh J
• Created: 06/Jul/12 16:29
• Updated: 13/Jun/22 17:00
• Resolved: 07/Jul/12 15:50

Description

I noticed that right now, under a bulkLoadHFiles call to an RS, we grab the HRegion write lock as soon as we determine that it is a multi-family bulk load we'll be attempting. The file copy from the caller's source FS is done after holding the lock.

This doesn't seem right. For instance, we had a recent use-case where the bulk load running cluster is a separate HDFS instance/cluster than the one that runs HBase and the transfers between these FSes can get slower than an intra-cluster transfer. Hence I think we should begin to hold the write lock only after we've got a successful destinationFS copy of the requested file, and thereby allow more write throughput to pass.

Does this sound reasonable to do?