Pig
  1. Pig
  2. PIG-3554

JobConf in UDFContext NULL when with @MonitoredUDF

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.11.1
    • Fix Version/s: None
    • Component/s: None
    • Labels:
    • Environment:

      hadoop 0.20.2
      jdk6
      back-end

    • Tags:
      udf udfcontext jobconf

      Description

      In the UDFContext the JobConf is always Null, when used with @MonitoredUDF

      1. TestJobConf.java
        0.8 kB
        Henning Kropp

        Activity

        Henning Kropp created issue -
        Hide
        Henning Kropp added a comment -

        A simple UDF to reproduce the Problem. It's a monitored UDF and when executed on the back-end it prints a out the following log message:

        2013-10-30 19:24:09,516 ERROR TestJobConf: java.util.concurrent.ExecutionException: java.lang.NullPointerException
        2013-10-30 19:24:10,388 ERROR TestJobConf: java.util.concurrent.ExecutionException: java.lang.NullPointerException
        2013-10-30 19:24:10,880 ERROR TestJobConf: java.util.concurrent.ExecutionException: java.lang.NullPointerException
        

        Comment out the MonitoredUDF Annotation and you'll this messages:

        Configuration: core-default.xml, core-site.xml, mapred-default.xml, mapred-site.xml, hdfs-default.xml, hdfs-site.xml, .....
        

        It should be possible to execute the attached UDF in the following manner:

        A = LOAD 'data' USING PigStorage();
        B = FOREACH A GENERATE TestJobConf(*);
        DUMP B;
        
        Show
        Henning Kropp added a comment - A simple UDF to reproduce the Problem. It's a monitored UDF and when executed on the back-end it prints a out the following log message: 2013-10-30 19:24:09,516 ERROR TestJobConf: java.util.concurrent.ExecutionException: java.lang.NullPointerException 2013-10-30 19:24:10,388 ERROR TestJobConf: java.util.concurrent.ExecutionException: java.lang.NullPointerException 2013-10-30 19:24:10,880 ERROR TestJobConf: java.util.concurrent.ExecutionException: java.lang.NullPointerException Comment out the MonitoredUDF Annotation and you'll this messages: Configuration: core- default .xml, core-site.xml, mapred- default .xml, mapred-site.xml, hdfs- default .xml, hdfs-site.xml, ..... It should be possible to execute the attached UDF in the following manner: A = LOAD 'data' USING PigStorage(); B = FOREACH A GENERATE TestJobConf(*); DUMP B;
        Henning Kropp made changes -
        Field Original Value New Value
        Attachment TestJobConf.java [ 12611266 ]

          People

          • Assignee:
            Unassigned
            Reporter:
            Henning Kropp
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:

              Development