[HDDS-8636] Containers in schema V1 fail for replication after the 1.3.0 upgrade

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.3.0
• Fix Version/s: 1.4.0
• Component/s: Ozone Datanode
• Labels: pull-request-available
• Target Version/s: 1.3.1
• Assignee: Attila Doroszlai
• Reporter: Kohei Sugihara
• Created: 17/May/23 10:20
• Updated: 25/May/23 03:55
• Resolved: 25/May/23 03:55

Description

Containers with the schema version V1 fail for replication with NullPointerException. I will attach the stack trace taken by the debugger. Source revision is our forked version (ozone-1.3 compatible).

 

"ContainerReplicationThread-30@14206" daemon prio=5 tid=0x939 nid=NA runnable
  java.lang.Thread.State: RUNNABLE
      at org.apache.hadoop.ozone.container.keyvalue.helpers.KeyValueContainerLocationUtil.getContainerDBFile(KeyValueContainerLocationUtil.java:110)
      at org.apache.hadoop.ozone.container.keyvalue.KeyValueContainer.getContainerDBFile(KeyValueContainer.java:824)
      at org.apache.hadoop.ozone.container.keyvalue.KeyValueContainer.populatePathFields(KeyValueContainer.java:230)
      at org.apache.hadoop.ozone.container.keyvalue.KeyValueHandler.populateContainerPathFields(KeyValueHandler.java:341)
      at org.apache.hadoop.ozone.container.keyvalue.KeyValueHandler.importContainer(KeyValueHandler.java:976)
      at org.apache.hadoop.ozone.container.ozoneimpl.ContainerController.importContainer(ContainerController.java:162)
      at org.apache.hadoop.ozone.container.replication.DownloadAndImportReplicator.importContainer(DownloadAndImportReplicator.java:83)
      at org.apache.hadoop.ozone.container.replication.DownloadAndImportReplicator.replicate(DownloadAndImportReplicator.java:121)
      at org.apache.hadoop.ozone.container.replication.MeasuredReplicator.replicate(MeasuredReplicator.java:77)
      at org.apache.hadoop.ozone.container.replication.ReplicationSupervisor$TaskRunner.run(ReplicationSupervisor.java:168)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
      at java.lang.Thread.run(Thread.java:829) 

 

The problem is containerData.getSchemaVersion() returns null. I think container schema V1 does not have the 'schemaVersion' field in a .container YAML so that this code will fail with NPE.

/**
 * Return containerDB File.
 */
public static File getContainerDBFile(KeyValueContainerData containerData) {
  if (containerData.getSchemaVersion().equals(OzoneConsts.SCHEMA_V3)) {
    return new File(containerData.getVolume().getDbParentDir(),
        OzoneConsts.CONTAINER_DB_NAME);
  }
  return getContainerDBFile(containerData.getMetadataPath(), containerData);
} 

 

Issue Links

• fixes: HDDS-8637 Add a null check for KeyValueContainerData#setSchemaVersion
• is fixed by: HDDS-8637 Add a null check for KeyValueContainerData#setSchemaVersion
• links to: GitHub Pull Request #4748