Details
-
Sub-task
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
3.4.0
-
Reviewed
Description
org.apache.hadoop.hdfs.testPipelineRecoveryOnRestartFailure failed with the stack like:
java.io.IOException: All datanodes [DatanodeInfoWithStorage[127.0.0.1:57448,DS-1b5f7e33-a2bf-4edc-9122-a74c995a99f5,DISK]] are bad. Aborting... at org.apache.hadoop.hdfs.DataStreamer.handleBadDatanode(DataStreamer.java:1667) at org.apache.hadoop.hdfs.DataStreamer.setupPipelineInternal(DataStreamer.java:1601) at org.apache.hadoop.hdfs.DataStreamer.setupPipelineForAppendOrRecovery(DataStreamer.java:1587) at org.apache.hadoop.hdfs.DataStreamer.processDatanodeOrExternalError(DataStreamer.java:1371) at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:674)
After tracing the root cause, this bug was introduced by HDFS-16534. Because the block GS of client may be smaller than DN when pipeline recovery failed.
Attachments
Attachments
Issue Links
- links to