[TEZ-2021] Tez tool to analyze shuffle performance in large clusters by mining task logs - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Open
Priority: Major
Resolution: Unresolved
Affects Version/s: None
Fix Version/s: None
Component/s: None
Labels:
None

Description

Tez tool to analyze shuffle performance in large clusters by mining task logs. Provide an easier way to visualize (heat chart) and identify bad nodes in large cluster.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

total_data_transferred.png
04/Feb/15 00:46
169 kB
Rajesh Balamohan
TEZ-2021.3.patch
27/Oct/15 17:49
27 kB
Rajesh Balamohan
TEZ-2021.2.patch
04/Feb/15 12:23
29 kB
Rajesh Balamohan
TEZ-2021.1.patch
04/Feb/15 00:51
28 kB
Rajesh Balamohan
no_of_times_contacted.png
04/Feb/15 00:46
199 kB
Rajesh Balamohan
avg_time_taken_to_download.png
04/Feb/15 00:46
119 kB
Rajesh Balamohan
avg_time_Taken_after_fix.png
04/Feb/15 12:23
123 kB
Rajesh Balamohan

Sub-Tasks

Refactor logIndividualFetchComplete() to be common for both shuffle-schedulers

Closed

Rajesh Balamohan

Activity

People

Assignee:: Rajesh Balamohan

Reporter:: Rajesh Balamohan

Votes:: 0 Vote for this issue

Watchers:: 6 Start watching this issue

Dates

Created:: 02/Feb/15 02:24

Updated:: 27/Oct/15 17:49