[PIG-3299] Provide support for LazyOutputFormat to avoid creating empty files - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: 0.11.1
Fix Version/s: 0.13.0
Component/s: None
Labels:
None

Hadoop Flags:

Reviewed
Release Note:
With this patch, the bundled Hadoop bump up to 1.0.4.

Description

LazyOutputFormat (~~HADOOP-4927~~) in hadoop is a wrapper to avoid creating part files if there is no records output. It would be good to add support for that by having a configuration in pig which wraps storeFunc.getOutputFormat() with LazyOutputFormat.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

PIG-3299.patch
04/Jan/14 01:16
7 kB
Lorand Bendig
PIG-3299-2.patch
31/Jan/14 20:09
8 kB
Daniel Dai
PIG-3299-3.patch
02/Feb/14 17:43
9 kB
Lorand Bendig
PIG-3299-4.patch
03/Feb/14 19:23
9 kB
Daniel Dai

Issue Links

is related to

MAPREDUCE-5839 Provide a boolean switch to enable LazyOutputFormat

Open

Activity

People

Assignee:: Lorand Bendig

Reporter:: Rohini Palaniswamy

Votes:: 0 Vote for this issue

Watchers:: 6 Start watching this issue

Dates

Created:: 29/Apr/13 13:58

Updated:: 07/Jul/14 18:08

Resolved:: 03/Feb/14 19:25