Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-38556

Disable Pandas usage logging for method calls inside @contextmanager functions

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: In Progress
    • Minor
    • Resolution: Unresolved
    • 3.2.1
    • None
    • PySpark
    • None

    Description

      Currently, calls inside @contextmanager functions are treated as external for with statements.

      For example, the below code records config.set_option calls inside ps.option_context(...)

      with ps.option_context("compute.ops_on_diff_frames", True):
          pass 

      We should disable usage logging for calls inside @contextmanager functions to improve accuracy of the usage data

       

      Attachments

        Activity

          People

            Unassigned Unassigned
            heyihong Yihong He
            Takuya Ueshin Takuya Ueshin
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: