[TEZ-4208] Pipelinesorter uses single SortSpan after spill - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 0.10.0
Component/s: None
Labels:
None

Description

Though it could have created multiple spans, tez always uses the first span after spill. It is quite possible that other spans are bigger compared to the first one, due to progressive space allocation. Fixing this would help in reducing the number of spills (depending on the jobs) and lesser load for indexcache entries (as lesser number of files have to be opened).

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

TEZ-4208.2.patch
04/Aug/20 11:08
5 kB
Rajesh Balamohan
TEZ-4208.1.patch
28/Jul/20 10:56
1.0 kB
Rajesh Balamohan
q67_sorter.log
28/Jul/20 10:54
112 kB
Rajesh Balamohan

Activity

People

Assignee:: Rajesh Balamohan

Reporter:: Rajesh Balamohan

Votes:: 0 Vote for this issue

Watchers:: 1 Start watching this issue

Dates

Created:: 28/Jul/20 10:54

Updated:: 25/Aug/20 15:00

Resolved:: 05/Aug/20 23:53