[PIG-3047] Check the size of a relation before adding it to distributed cache in Replicated join - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 0.12.1
Component/s: None
Labels:
None

Description

Right now if someone makes a mistake and put the large relation last, Pig will copy a huge file into distributed cache and it will take a long time before the job eventually fails. It would be better to check before copying the relation that it is of reasonable size.
<1 GB ?

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

fix_pig_3047_1.diff
20/Oct/13 15:02
6 kB
Ido Hadanny
fix_pig_3047.diff
16/Oct/13 13:56
6 kB
Ido Hadanny
PIG-3047-2.patch
26/Oct/13 01:30
12 kB
Aniket Namadeo Mokashi
PIG-3047-3.patch
28/Oct/13 23:02
14 kB
Aniket Namadeo Mokashi

Activity

People

Assignee:: Aniket Namadeo Mokashi

Reporter:: Julien Le Dem

Votes:: 0 Vote for this issue

Watchers:: 7 Start watching this issue

Dates

Created:: 13/Nov/12 20:53

Updated:: 15/Apr/14 20:44

Resolved:: 29/Oct/13 00:16