[HDDS-2268] Incorrect container checksum upon downgrade

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Component/s: Ozone Datanode
• Labels: upgrade
• Target Version/s: 1.2.0
• Assignee: Aravindan Vijayan
• Reporter: Attila Doroszlai
• Created: 08/Oct/19 20:45
• Updated: 26/Jul/21 18:05
• Resolved: 26/Jul/21 18:05

Description

Container file checksum is calculated based on all YAML fields in a given Ozone version. If the same container file is used in older Ozone, which has fewer fields, the expected checksum will be different.

Example: origin pipeline ID and origin node ID were added for HDDS-837 in Ozone 0.4.0. Starting Ozone 0.3.0 with the same data results in checksum error.

datanode_1  | ... ERROR ContainerReader:166 - Failed to parse ContainerFile for ContainerID: 1
datanode_1  | org.apache.hadoop.hdds.scm.container.common.helpers.StorageContainerException: Container checksum error for ContainerID: 1.
datanode_1  | Stored Checksum: 7a6ec508d6e3796c5fe5fd52574b3d3437b0a0eaa4e053f7a96a5e39f4abb374
datanode_1  | Expected Checksum: fee023a02d3ced2f7b0b42c116cce5f03da6b57b29965ca878dc46d1213230b6
datanode_1  | 	at org.apache.hadoop.ozone.container.common.helpers.ContainerUtils.verifyChecksum(ContainerUtils.java:259)
datanode_1  | 	at org.apache.hadoop.ozone.container.keyvalue.helpers.KeyValueContainerUtil.parseKVContainerData(KeyValueContainerUtil.java:165)
datanode_1  | 	at org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader.verifyContainerData(ContainerReader.java:180)
datanode_1  | 	at org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader.verifyContainerFile(ContainerReader.java:164)
datanode_1  | 	at org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader.readVolume(ContainerReader.java:142)

Issue Links

• relates to: HDDS-5129 Support new field in datanode container descriptors