[HDFS-16422] Fix thread safety of EC decoding during concurrent preads

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 3.3.0, 3.3.1
• Fix Version/s: 3.4.0, 3.2.3, 3.3.3
• Component/s: dfsclient, ec, erasure-coding
• Labels: pull-request-available
• Target Version/s: 3.4.0, 3.3.3
• Hadoop Flags: Reviewed
• Assignee: daimin
• Reporter: daimin
• Due: 13/Jan/22
• Created: 13/Jan/22 06:35
• Updated: 27/Jan/24 01:30
• Resolved: 18/Apr/22 14:10

Description

Reading data on an erasure-coded file with missing replicas(internal block of block group) will cause online reconstruction: read dataUnits part of data and decode them into the target missing data. Each DFSStripedInputStream object has a RawErasureDecoder object, and when we doing pread concurrently, RawErasureDecoder.decode will be invoked concurrently too. RawErasureDecoder.decode is not thread safe, as a result of that we get wrong data from pread occasionally.

Issue Links

• is depended upon by: HADOOP-18198 Release Hadoop 3.3.3: hadoop-3.3.2 with some fixes
• links to: GitHub Pull Request #3881