[PIG-2652] Skew join and order by don't trigger reducer estimation - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 0.11
Component/s: None
Labels:
None

Release Note:
Fix how reducers are estimated for skew join and order operators.

Description

If neither PARALLEL, default parallel or mapred.reduce.tasks are set, the number of reducers is not estimated based on input size for skew joins or order by. Instead, these jobs get only 1 reducer.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

PIG-2652_1.patch
13/Apr/12 23:54
0.8 kB
William W. Graham Jr
PIG-2652_2.patch
15/Apr/12 22:44
2 kB
Daniel Dai
PIG-2652_3_10.patch
16/Apr/12 00:44
9 kB
Daniel Dai
PIG-2652_3.patch
16/Apr/12 00:20
10 kB
Daniel Dai
PIG-2652_4.patch
18/Apr/12 19:30
9 kB
Dmitriy V. Ryaboy
PIG-2652_5.patch
18/Apr/12 22:29
11 kB
Dmitriy V. Ryaboy
PIG-2652_6.patch
23/Apr/12 17:22
15 kB
Dmitriy V. Ryaboy
PIG-2652_7.patch
26/Apr/12 08:19
24 kB
Dmitriy V. Ryaboy

Issue Links

relates to

PIG-2779 Refactoring the code for setting number of reducers

Closed

Activity

People

Assignee:: Dmitriy V. Ryaboy

Reporter:: William W. Graham Jr

Votes:: 0 Vote for this issue

Watchers:: 6 Start watching this issue

Dates

Created:: 13/Apr/12 23:43

Updated:: 22/Feb/13 04:53

Resolved:: 28/Jun/12 20:51