[TEZ-1526] LoadingCache for TezTaskID slow for large jobs - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 0.9.0
Component/s: None
Labels:
- performance

Target Version/s:

0.7.2

Description

Using the LoadingCache with default builder settings. 100,000 TezTaskIDs are created in 10 seconds on my setup. With a LoadingCache initialCapacity of 10,000 they are created in 300 ms. With no LoadingCache, they are created in 10 ms. A test case in attached to illustrate the condition I would like to be sped up.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

100000-TezTaskIDs.patch
01/Sep/14 03:44
0.8 kB
Jonathan Turner Eagles
hamlet.txt
13/Mar/17 18:23
206 kB
Jonathan Turner Eagles
TEZ-1526.3.patch
07/May/15 20:37
2 kB
Jonathan Turner Eagles
TEZ-1526.4.patch
07/May/15 21:13
5 kB
Jonathan Turner Eagles
TEZ-1526.5.patch
15/Apr/16 02:32
13 kB
Jonathan Turner Eagles
TEZ-1526.6.patch
24/Feb/17 23:28
19 kB
Jonathan Turner Eagles
TEZ-1526.7.patch
03/Mar/17 23:19
22 kB
Jonathan Turner Eagles
TEZ-1526.8.patch
13/Mar/17 16:12
21 kB
Jonathan Turner Eagles
TEZ-1526.memory.test.patch
13/Mar/17 16:12
3 kB
Jonathan Turner Eagles
TEZ-1526-v1.patch
01/Sep/14 05:42
6 kB
Jonathan Turner Eagles
TEZ-1526-v2.patch
10/Sep/14 04:17
10 kB
Jonathan Turner Eagles

Issue Links

is related to

YARN-6768 Improve performance of yarn api record toString and fromString

Resolved

Activity

People

Assignee:: Jonathan Turner Eagles

Reporter:: Jonathan Turner Eagles

Votes:: 0 Vote for this issue

Watchers:: 3 Start watching this issue

Dates

Created:: 01/Sep/14 03:41

Updated:: 30/Dec/21 04:07

Resolved:: 14/Mar/17 20:13