[PIG-4819] RANDOM() udf can lead to missing or redundant records

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.16.0
• Hadoop Flags: Reviewed
• Assignee: Koji Noguchi
• Reporter: Koji Noguchi
• Created: 01/Mar/16 20:18
• Updated: 08/Jun/16 20:48
• Resolved: 03/Mar/16 20:58

Description

When RANDOM() value is used for grouping/distinct/etc, it breaks the mapreduce rule and can lead to redundant or missing records.

Some discussion can be found in
https://issues.apache.org/jira/browse/PIG-3257?focusedCommentId=13669195#comment-13669195

We should make RANDOM less random so that it'll produce the same sequence of random values from the task retries.