Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-4908

JythonFunction refers to Oozie launcher script absolute path

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.16.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      We had a scenario where a user had multiple udfs all named udfs.py. JythonFunction was referring to the absolute localized path of udfs.py in the Oozie launcher. Tasks which ran on a node and which had a different version of udfs.py localized to same path (hashcode conflict) as the Oozie launcher failed. We should be referring to relative path of the files. The current code check is the canonical path starts with cwd, but that does not work as the files are downloaded to a different location and symlinked to the current working directory of the task attempt.

        Attachments

        1. PIG-4908-3-fixtest.patch
          3 kB
          Rohini Palaniswamy
        2. PIG-4908-2-fixtest.patch
          0.7 kB
          Rohini Palaniswamy
        3. PIG-4908-1.patch
          2 kB
          Rohini Palaniswamy

          Activity

            People

            • Assignee:
              rohini Rohini Palaniswamy
              Reporter:
              rohini Rohini Palaniswamy
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: