[HADOOP-11506] Configuration variable expansion regex expensive for long values - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 2.6.1, 2.7.0, 3.0.0-alpha1
Component/s: conf
Labels:
- 2.6.1-candidate

Target Version/s:

2.7.0
Hadoop Flags:

Reviewed

Description

Profiling several large Hadoop jobs, we discovered that a surprising amount of time was spent inside Configuration.get, more specifically, in regex matching caused by the substituteVars call.

Attachments

Options
- Sort By Name
- Sort By Date
- Ascending
- Descending

Attachments

HADOOP-11506.004.patch
02/Feb/15 21:06
9 kB
Gera Shegalov
HADOOP-11506.003.patch
01/Feb/15 01:00
9 kB
Gera Shegalov
HADOOP-11506.002.patch
28/Jan/15 10:59
6 kB
Gera Shegalov
HADOOP-11506.001.patch
24/Jan/15 01:19
4 kB
Gera Shegalov

Issue Links

breaks

HADOOP-13699 Configuration does not substitute multiple references to the same var

Resolved

relates to

HADOOP-463 variable expansion in Configuration

Closed

Activity

People

Assignee:

Gera Shegalov

Reporter:

Dmitriy V. Ryaboy

Votes:

0 Vote for this issue

Watchers:

15 Start watching this issue

Dates

Created:

22/Jan/15 22:40

Updated:

10/Oct/16 19:17

Resolved:

06/Feb/15 10:02