[PIG-3588] Integrate Pig with Morphlines.

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.12.0
• Component/s: internal-udfs
• Assignee: Unassigned
• Reporter: Wolfgang Hoschek
• Created: 25/Nov/13 19:13
• Updated: 25/Nov/13 19:13

Description

Integrate Pig with Morphlines.

Specifically, add support to Pig to call a morphline as a UDTF to leverage existing morphlines ETL functionality. Often, some flexible massaging needs to happen to get the input data into the shape that's desired, and morphline logic helps do this in a user-friendly, pluggable, efficient, pipelined manner. This issue basically boils down to transforming an input tuple into a morphline record, feeding the record into the morphline processing API, and finally converting zero or more morphline output records into corresponding Pig tuples.

Some background is here:

http://cloudera.github.io/cdk/docs/current/cdk-morphlines/morphlinesReferenceGuide.html

http://cloudera.github.io/cdk/docs/current/cdk-morphlines/index.html