[PIG-3047] Check the size of a relation before adding it to distributed cache in Replicated join - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 0.12.1
Component/s: None
Labels:
None

Description

Right now if someone makes a mistake and put the large relation last, Pig will copy a huge file into distributed cache and it will take a long time before the job eventually fails. It would be better to check before copying the relation that it is of reasonable size.
<1 GB ?

Attachments

Options
- Sort By Name
- Sort By Date
- Ascending
- Descending

Attachments

PIG-3047-3.patch
28/Oct/13 23:02
14 kB
Aniket Namadeo Mokashi
PIG-3047-2.patch
26/Oct/13 01:30
12 kB
Aniket Namadeo Mokashi
fix_pig_3047_1.diff
20/Oct/13 15:02
6 kB
Ido Hadanny
fix_pig_3047.diff
16/Oct/13 13:56
6 kB
Ido Hadanny

Activity

People

Assignee:

Aniket Namadeo Mokashi

Reporter:

Julien Le Dem

Votes:

0 Vote for this issue

Watchers:

7 Start watching this issue

Dates

Created:

13/Nov/12 20:53

Updated:

15/Apr/14 20:44

Resolved:

29/Oct/13 00:16