[CRUNCH-351] Improve performance of Shard#shard on large records - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 0.10.0, 0.8.3
Component/s: None
Labels:
None

Description

This avoids sorting on the input data, which may be long and make
shuffle phase slow. The improvement is to sort on pseudo-random numbers.

Attachments

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

crunch-351.patch
22/Feb/14 10:38
4 kB
Chao Shi
crunch-351-v2.patch
23/Feb/14 11:10
4 kB
Chao Shi

Activity

People

Assignee:: Chao Shi

Reporter:: Chao Shi

Votes:: 0 Vote for this issue

Watchers:: 4 Start watching this issue

Dates

Created:: 22/Feb/14 10:37

Updated:: 20/Jun/14 03:49

Resolved:: 29/Mar/14 17:03