[HADOOP-11506] Configuration variable expansion regex expensive for long values

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.6.1, 2.7.0, 3.0.0-alpha1
• Component/s: conf
• Labels: 2.6.1-candidate
• Target Version/s: 2.7.0
• Hadoop Flags: Reviewed
• Assignee: Gera Shegalov
• Reporter: Dmitriy V. Ryaboy
• Created: 22/Jan/15 22:40
• Updated: 10/Oct/16 19:17
• Resolved: 06/Feb/15 10:02

Description

Profiling several large Hadoop jobs, we discovered that a surprising amount of time was spent inside Configuration.get, more specifically, in regex matching caused by the substituteVars call.

Issue Links

• breaks: HADOOP-13699 Configuration does not substitute multiple references to the same var
• relates to: HADOOP-463 variable expansion in Configuration