Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-20970

Deprecate TaskMetrics._updatedBlockStatuses

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Incomplete
    • Affects Version/s: 2.2.0
    • Fix Version/s: None
    • Component/s: Spark Core
    • Labels:
    • Target Version/s:

      Description

      TaskMetrics._updatedBlockStatuses isn't used anywhere internally by spark. It could be used by users though since its exposed by SparkListenerTaskEnd. We made it configurable to turn off the tracking of it since it uses a lot of memory in https://issues.apache.org/jira/browse/SPARK-20923. That config is still true for backwards compatibility. We should turn that to false in next release and deprecate that api altogether.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                tgraves Thomas Graves
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: