[HBASE-13585] HRegionFileSystem#splitStoreFile() finishes without closing the file handle in some situation

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.1.0, 1.2.0, 2.0.0
• Fix Version/s: 1.1.0, 0.98.13, 1.0.2, 2.0.0
• Component/s: regionserver

Environment

Windows

• Hadoop Flags: Reviewed
• Flags: Patch
• Assignee: Stephen Yuan Jiang
• Reporter: Stephen Yuan Jiang
• Created: 28/Apr/15 18:22
• Updated: 04/Jul/15 12:52
• Resolved: 29/Apr/15 00:35

Description

HRegionFileSystem#splitStoreFile() opens a reader and does not close it in some situation.

In Windows, TestSplitTransaction#testWholesomeSplit() consistently failed due to open file handle left at the end of test

Failed delete of C:/hbase/hbase-server/target/test-data/b470118c-978a-4915-8a12-b29b2c966beb/org.apache.hadoop.hbase.regionserver.TestSplitTransaction/data/default/table/a578b53b3c3a947c5f617c51ccb982cf

Stacktrace

java.io.IOException: Failed delete of C:/hbase/hbase-server/target/test-data/b470118c-978a-4915-8a12-b29b2c966beb/org.apache.hadoop.hbase.regionserver.TestSplitTransaction/data/default/table/a578b53b3c3a947c5f617c51ccb982cf
	at org.apache.hadoop.hbase.regionserver.TestSplitTransaction.teardown(TestSplitTransaction.java:106)


Standard Error

2015-04-26 12:57:43,863 WARN  [main] fs.FileUtil(187): Failed to delete file or dir [C:\hbase\hbase-server\target\test-data\b470118c-978a-4915-8a12-b29b2c966beb\org.apache.hadoop.hbase.regionserver.TestSplitTransaction\data\default\table\a578b53b3c3a947c5f617c51ccb982cf\info\.0519debf2b934245a9a8c3d7cfc0f51d.crc]: it still exists.
2015-04-26 12:57:43,864 WARN  [main] fs.FileUtil(187): Failed to delete file or dir [C:\hbase\hbase-server\target\test-data\b470118c-978a-4915-8a12-b29b2c966beb\org.apache.hadoop.hbase.regionserver.TestSplitTransaction\data\default\table\a578b53b3c3a947c5f617c51ccb982cf\info\.45fd4e817ce64759abc6e982a5c0b830.crc]: it still exists.
2015-04-26 12:57:43,865 WARN  [main] fs.FileUtil(187): Failed to delete file or dir [C:\hbase\hbase-server\target\test-data\b470118c-978a-4915-8a12-b29b2c966beb\org.apache.hadoop.hbase.regionserver.TestSplitTransaction\data\default\table\a578b53b3c3a947c5f617c51ccb982cf\info\.8e6bc752c9fc414abfe085f8959dce94.crc]: it still exists.
2015-04-26 12:57:43,870 WARN  [main] fs.FileUtil(187): Failed to delete file or dir [C:\hbase\hbase-server\target\test-data\b470118c-978a-4915-8a12-b29b2c966beb\org.apache.hadoop.hbase.regionserver.TestSplitTransaction\data\default\table\a578b53b3c3a947c5f617c51ccb982cf\info\0519debf2b934245a9a8c3d7cfc0f51d]: it still exists.
2015-04-26 12:57:43,870 WARN  [main] fs.FileUtil(187): Failed to delete file or dir [C:\hbase\hbase-server\target\test-data\b470118c-978a-4915-8a12-b29b2c966beb\org.apache.hadoop.hbase.regionserver.TestSplitTransaction\data\default\table\a578b53b3c3a947c5f617c51ccb982cf\info\45fd4e817ce64759abc6e982a5c0b830]: it still exists.
2015-04-26 12:57:43,871 WARN  [main] fs.FileUtil(187): Failed to delete file or dir [C:\hbase\hbase-server\target\test-data\b470118c-978a-4915-8a12-b29b2c966beb\org.apache.hadoop.hbase.regionserver.TestSplitTransaction\data\default\table\a578b53b3c3a947c5f617c51ccb982cf\info\8e6bc752c9fc414abfe085f8959dce94]: it still exists.

HBASE-8300 tried to fix this issue by adding 'StoreFile#closeReader()' call. However, after the open reader calls, there are 4 'return null' calls before the close call. When I run TestSplitTransaction#testWholesomeSplit(), 2 of the 'return null' calls were hit multiple times (which means the opened file is not closed).

The fix needs to make sure that StoreFile#closeReader() is called in all situation.

Issue Links

• is related to: HBASE-8300 TestSplitTransaction fails to delete files due to open handles left when region is split