Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-18576

Expose basic TaskContext info in PySpark

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.2.0
    • Component/s: PySpark
    • Labels:
      None

      Description

      Currently the TaskContext info isn't exposed in PySpark. While we don't need to expose the full TaskContext information in Python it would make sense to expose the public APIs in Python for users who are doing custom logging or job handling with task id or retry attempt.

        Attachments

          Activity

            People

            • Assignee:
              holden Holden Karau
              Reporter:
              holden Holden Karau
            • Votes:
              1 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: