[HADOOP-13699] Configuration does not substitute multiple references to the same var

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 3.0.0-alpha1
• Fix Version/s: 3.0.0-alpha2
• Component/s: conf
• Target Version/s: 3.0.0-alpha2
• Hadoop Flags: Incompatible change

Release Note

This changes the config var cycle detection introduced in 3.0.0-alpha1 by HADOOP-6871 such that it detects single-variable but not multi-variable loops. This also fixes resolution of multiple specifications of the same variable in a config value.

• Assignee: Andrew Wang
• Reporter: Andrew Wang
• Created: 07/Oct/16 23:06
• Updated: 10/Oct/16 20:01
• Resolved: 10/Oct/16 19:19

Description

Config var loop detection was originally introduced by HADOOP-6871. Due to cycle detection changes in the trunk patch for HADOOP-11506, resolution for multiple references to the same variable no longer resolved, e.g.

somekey = "${otherkey} ${otherkey}"

This loop detection business is fragile, expensive, and not in branch-2, so let's reduce it.

Issue Links

• is broken by: HADOOP-11506 Configuration variable expansion regex expensive for long values
• supercedes: HADOOP-6871 When the value of a configuration key is set to its unresolved form, it causes the IllegalStateException in Configuration.get() stating that substitution depth is too large.