Details
-
New Feature
-
Status: Closed
-
Major
-
Resolution: Won't Fix
-
None
-
None
-
None
Description
Min Zhou has provided this suggestion (Issue #88 on GitHub):
Lead is an analytic function like Oracle's Lead function. It provides access to more than one tuple of a bag at the same time without a self join. Given a bag of tuple returned from a query, LEAD provides access to a tuple at a given physical offset beyond that position. Generates pairs of all items in a bag.
If you do not specify offset, then its default is 1. Null is returned if the offset goes beyond the scope of the bag.
Example 1:
register ba-pig-0.1.jar define Lead datafu.pig.bags.Lead('2'); -- INPUT: ({(1),(2),(3),(4)}) data = LOAD 'input' AS (data: bag {T: tuple(v:INT)}); describe data; -- OUTPUT: ({((1),(2),(3)),((2),(3),(4)),((3),(4),),((4),,)}) -- OUTPUT SCHEMA: data2: {lead_data: {(elem0: (v: int),elem1: (v: int),elem2: (v: int))}} data2 = FOREACH data GENERATE Lead(data); describe data2; DUMP data2;
Example 2
register ba-pig-0.1.jar define Lead datafu.pig.bags.Lead(); -- INPUT: ({(10,{(1),(2),(3)}),(20,{(4),(5),(6)}),(30,{(7),(8)}),(40,{(9),(10),(11)}),(50,{(12),(13),(14),(15)})}) data = LOAD 'input' AS (data: bag {T: tuple(v1:INT,B: bag{T: tuple(v2:INT)})}); --describe data; -- OUPUT: ({((10,{(1),(2),(3)}),(20,{(4),(5),(6)})),((20,{(4),(5),(6)}),(30,{(7),(8)})),((30,{(7),(8)}),(40,{(9),(10),(11)})),((40,{(9),(10),(11)}),(50,{(12),(13),(14),(15)})),((50,{(12),(13),(14),(15)}),)}) data2 = FOREACH data GENERATE Lead(data); --describe data2; DUMP data2;