Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-4059 Pig on Spark
  3. PIG-4232

UDFContext is not initialized in executors when running on Spark cluster

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: spark-branch
    • Component/s: spark
    • Labels:
      None

      Description

      UDFContext is used in lot of features across pig code base. For example its used in PigStorage to pass columns information between the frontend and the backend code. https://github.com/apache/pig/blob/spark/src/org/apache/pig/builtin/PigStorage.java#L246-L247

        Attachments

        1. pouserfunc.readObject.stacktrace.txt
          18 kB
          liyunzhang
        2. PIG-4232.patch
          0.7 kB
          liyunzhang
        3. PIG-4232_6.patch
          13 kB
          liyunzhang
        4. PIG-4232_5.patch
          13 kB
          liyunzhang
        5. PIG-4232_4.patch
          13 kB
          liyunzhang
        6. PIG-4232_3.patch
          12 kB
          liyunzhang
        7. PIG-4232_2.patch
          5 kB
          Praveen Rachabattuni
        8. PIG-4232_1.patch
          7 kB
          liyunzhang

          Issue Links

            Activity

              People

              • Assignee:
                kellyzly liyunzhang
                Reporter:
                praveenr019 Praveen Rachabattuni
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: