[HDFS-9821] HDFS configuration should accept friendly time units

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.8.0
• Component/s: datanode, namenode
• Target Version/s: 3.5.0
• Assignee: Xiaobing Zhou
• Reporter: Arpit Agarwal
• Created: 17/Feb/16 22:18
• Updated: 04/Jan/24 07:51

Description

HDFS configuration keys that define time intervals use units inconsistently (Hours, seconds, milliseconds).

Not all keys have the unit as part of their name. Related keys may use different units e.g. dfs.blockreport.intervalMsec accepts msec while dfs.blockreport.initialDelay accepts seconds. Milliseconds is rarely useful as a time unit which makes these values hard to parse when reading config files.

We can either

1. Let existing keys use friendly units e.g. 100ms, 60s, 5m, 1d, 6w etc. This can be done compatibly since there will be no conflict with existing valid configuration. If no suffix is specified just default to the current time unit.
2. Just deprecate the existing keys and define new ones that accept friendly units.

We continue to use fine-grained time units (usually ms) internally in code and also accept "ms" option for tests.

Issue Links

• is related to: HADOOP-12848 Configuration.ParsedTimeDuration.unitFor(String) should do more careful parsing
• is related to: HADOOP-8608 Add Configuration API for parsing time durations

Sub-Tasks

• HDFS configuration should accept time units: Resolved
• Change defaults in hdfs-site.xml to match timeunit type: Resolved