[PIG-2101] Registering a Python function in a directory other than the current working directory fails

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.8.1
• Component/s: impl
• Assignee: Unassigned
• Reporter: Alan Gates
• Created: 31/May/11 23:04
• Updated: 06/Apr/12 17:57

Description

In MapReduce mode, if the register command references a directory other than the current one, executing the Python UDF on the backend fails with: Deserialization error: could not instantiate 'org.apache.pig.scripting.jython.JythonFunction' with arguments '[../udfs/python/production.py, production]'

I assume it is using the path on the backend to try to locate the UDF.

The script is:

register '../udfs/python/production.py' using jython as bballudfs;
players  = load 'baseball' as (name:chararray, team:chararray,
                pos:bag{t:(p:chararray)}, bat:map[]);
nonnull  = filter players by bat#'slugging_percentage' is not null and
                bat#'on_base_percentage' is not null;
calcprod = foreach nonnull generate name, bballudfs.production(
                (float)bat#'slugging_percentage',
                (float)bat#'on_base_percentage');
dump calcprod;

Issue Links

• is duplicated by: PIG-2542 Jython UDFs only work if the python file is in the current directory