[PIG-1824] Support import modules in Jython UDF

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.8.0, 0.9.0
• Fix Version/s: 0.10.0
• Hadoop Flags: Reviewed

Release Note

module import state is determined before and after user code is executed. The resolved modules are inspected and added to the pigContext, then they are added to the job jar.

this patch addresses the following import modes:
- import re, which will (if configured) find re on the filesystem in the jython install root
- import foo (which can import bar), this works now provided bar is resolvable JYTHON_HOME, JYTHONPATH, curdir, etc.
- from pkg import *, which works when the cachedir is writable
- import non.jvm.class, which works when the cachedir is writable
- the directly imported module may use schema decorators, but recursively imported modules cannot until PIG-1943 is addressed

• Tags: jython, import
• Assignee: Woody Anderson
• Reporter: Richard Ding
• Created: 26/Jan/11 02:22
• Updated: 06/May/13 17:59
• Resolved: 20/May/11 20:04

Description

Currently, Jython UDF script doesn't support Jython import statement as in the following example:

#!/usr/bin/python

import re
@outputSchema("word:chararray")
def resplit(content, regex, index):
        return re.compile(regex).split(content)[index]

Can Pig automatically locate the Jython module file and ship it to the backend? Or should we add a ship clause to let user explicitly specify the module to ship?

Issue Links

• is related to: PIG-2433 Jython import module not working if module path is in classpath
• relates to: PIG-2665 Bundled Jython jar in Pig 0.10.0-RC breaks module import in Python scripts with embedded Pig Latin