[HDDS-815] Rename Ozone/HDDS config keys prefixed with 'dfs' - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Resolved
Priority: Major
Resolution: Done
Affects Version/s: None
Fix Version/s: 1.5.0
Component/s: None
Labels:
- newbie
- pull-request-available

Target Version/s:

1.5.0

Description

The following Ozone config keys are prefixed with dfs. which is the prefix used by HDFS. Instead we should prefix them with hdds., since they belong to HDDS.

dfs.container.chunk.write.sync
dfs.container.ipc
dfs.container.ipc.random.port
dfs.container.ratis.admin.port
dfs.container.ratis.datanode.storage.dir
dfs.container.ratis.datastream.enabled
dfs.container.ratis.datastream.port
dfs.container.ratis.datastream.random.port
dfs.container.ratis.enabled
dfs.container.ratis.ipc
dfs.container.ratis.ipc.random.port
dfs.container.ratis.leader.pending.bytes.limit
dfs.container.ratis.log.appender.queue.byte-limit
dfs.container.ratis.log.appender.queue.num-elements
dfs.container.ratis.log.purge.gap
dfs.container.ratis.log.queue.byte-limit
dfs.container.ratis.log.queue.num-elements
dfs.container.ratis.num.container.op.executors
dfs.container.ratis.num.write.chunk.threads.per.volume
dfs.container.ratis.replication.level
dfs.container.ratis.rpc.type
dfs.container.ratis.segment.preallocated.size
dfs.container.ratis.segment.size
dfs.container.ratis.server.port
dfs.container.ratis.statemachinedata.sync.retries
dfs.container.ratis.statemachinedata.sync.timeout
dfs.container.ratis.statemachine.max.pending.apply-transactions
dfs.ratis.leader.election.minimum.timeout.duration
dfs.ratis.server.retry-cache.timeout.duration
dfs.ratis.snapshot.threshold

Additionally, we should append .port to the following keys:

dfs.container.ipc should be changed to hdds.container.ipc.port
dfs.container.ratis.ipc should be changed to hdds.container.ratis.ipc.port

When changing config keys, backward compatibility must be ensured by adding DeprecationDelta for the pairs of old/new keys in OzoneConfiguration:

https://github.com/apache/ozone/blob/60bcdafa73affe68d3ebed5a74e7b54a414efcf4/hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/conf/OzoneConfiguration.java#L313-L328

  private static void addDeprecatedKeys() {
    Configuration.addDeprecations(new DeprecationDelta[]{
        new DeprecationDelta("ozone.datanode.pipeline.limit",
            ScmConfigKeys.OZONE_DATANODE_PIPELINE_LIMIT),
        new DeprecationDelta(HDDS_DATANODE_RATIS_PREFIX_KEY + "."
           + RaftServerConfigKeys.PREFIX + "." + "rpcslowness.timeout",
           HDDS_DATANODE_RATIS_PREFIX_KEY + "."
           + RaftServerConfigKeys.PREFIX + "." + "rpc.slowness.timeout"),
        new DeprecationDelta("dfs.datanode.keytab.file",
            DFSConfigKeysLegacy.DFS_DATANODE_KERBEROS_KEYTAB_FILE_KEY),
        new DeprecationDelta("ozone.scm.chunk.layout",
            ScmConfigKeys.OZONE_SCM_CONTAINER_LAYOUT_KEY),
        new DeprecationDelta("hdds.datanode.replication.work.dir",
            OZONE_CONTAINER_COPY_WORKDIR)
    });
  }