[HDFS-16272] Int overflow in computing safe length during EC block recovery

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 3.3.0, 3.3.1
• Fix Version/s: 3.4.0, 3.2.3, 3.3.2
• Component/s: ec, erasure-coding
• Labels: pull-request-available

Environment

Cluster settings: EC RS-8-2-256k, Block Size 1GiB.

• Assignee: daimin
• Reporter: daimin
• Created: 13/Oct/21 07:43
• Updated: 11/Feb/24 23:12
• Resolved: 18/Oct/21 11:42

Description

There exists an int overflow problem in StripedBlockUtil#getSafeLength, which will produce a negative or zero length:
1. With negative length, it fails to the later >=0 check, and will crash the BlockRecoveryWorker thread, which make the lease recovery operation unable to finish.
2. With zero length, it passes the check, and directly truncate the block size to zero, leads to data lossing.

If you are using any of the default EC policies (3-2, 6-3 or 10-4) and the default HDFS block size of 128MB, then you will not be impacted by this issue.

To be impacted, the EC dataNumber * blockSize has to be larger than the Java max int of 2,147,483,647.

For example 10-4 is 10 * 134217728 = 1,342,177,280 which is OK.

However 10-4 with 256MB blocks is 2,684,354,560 which overflows the INT and causes the problem.

Issue Links

• links to: GitHub Pull Request #3548