[HBASE-27547] Close store file readers after region warmup

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.6.0, 3.0.0-alpha-4, 2.4.16, 2.5.3
• Component/s: regionserver
• Hadoop Flags: Reviewed
• Assignee: Eungsop Yoo
• Reporter: Eungsop Yoo
• Created: 29/Dec/22 09:49
• Updated: 29/Jan/23 19:48
• Resolved: 28/Jan/23 15:46

Description

We recently upgraded one of our clusters from version 1.4.13 to 2.4.15. After that we noticed that the number of open FD was increased 10 times more than before, from 1.3k to 15k. Each RS has about 1000 regions.

 

I thouht that it is OK if the number of open FD keeps steady. But the number was increased when regions were balanced. I found that there is a bug related to the region warmup process, which leaks opened FDs, and I fixed it. Please review the PR.

 

But I still have a question about why HBase 2.4 opens much more than 1.4. I am trying to find the answer, but not yet. Is there any person who can answer the question?

Issue Links

• links to: GitHub Pull Request #4942