Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-4059 Pig on Spark
  3. PIG-4232

UDFContext is not initialized in executors when running on Spark cluster

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • spark-branch
    • spark
    • None

    Description

      UDFContext is used in lot of features across pig code base. For example its used in PigStorage to pass columns information between the frontend and the backend code. https://github.com/apache/pig/blob/spark/src/org/apache/pig/builtin/PigStorage.java#L246-L247

      Attachments

        1. PIG-4232.patch
          0.7 kB
          liyunzhang
        2. PIG-4232_1.patch
          7 kB
          liyunzhang
        3. pouserfunc.readObject.stacktrace.txt
          18 kB
          liyunzhang
        4. PIG-4232_2.patch
          5 kB
          Praveen Rachabattuni
        5. PIG-4232_3.patch
          12 kB
          liyunzhang
        6. PIG-4232_4.patch
          13 kB
          liyunzhang
        7. PIG-4232_5.patch
          13 kB
          liyunzhang
        8. PIG-4232_6.patch
          13 kB
          liyunzhang

        Issue Links

          blocks

          Sub-task - The sub-task of the issue PIG-4207 Make python udfs work with Spark

          • Major - Major loss of function.
          • Closed
          is related to

          Sub-task - The sub-task of the issue PIG-4228 SchemaTupleBackend error when working on a Spark 1.1.0 cluster

          • Major - Major loss of function.
          • Open
          relates to

          Sub-task - The sub-task of the issue PIG-4207 Make python udfs work with Spark

          • Major - Major loss of function.
          • Closed

          Sub-task - The sub-task of the issue PIG-4209 Make stream work with Spark

          • Major - Major loss of function.
          • Closed

          Activity

            People

              kellyzly liyunzhang
              praveenr019 Praveen Rachabattuni
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: