Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-4913

Reduce jython function initiation during compilation

    Details

    • Type: Improvement
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      While investigating PIG-4908, saw that ScriptEngine.getScriptAsStream was invoked way too many times during compilation phase for a simple script.

      sleep.py
      #!/usr/bin/python
      
      import time;
      
      @outputSchema("sltime:int")
      def sleep(num):
          if num == 1:
              print "Sleeping for %d minutes" % num;
              time.sleep(num * 60);
          return num;
      
      sleep.pig
      register 'sleep.py' using jython;
      
      A = LOAD '/tmp/sleepdata' as (f1:int);
      B = FOREACH A generate $0, sleep($0);
      STORE B into '/tmp/tezout';
      

        Attachments

        1. PIG-4913.patch
          3 kB
          Adam Szita
        2. PIG-4913.2.patch
          10 kB
          Adam Szita

          Activity

            People

            • Assignee:
              szita Adam Szita
              Reporter:
              rohini Rohini Palaniswamy
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: