Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-43892

Add support for df[|] suggest column

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.5.0
    • 3.5.0
    • PySpark
    • None

    Description

      We don't have a keys method for pyspark dataframe, even though we support df['column name']

      And when we want to get a suggestion for df[|], we can't get anything. So this could be an improvement for autocomplete engine that uses keys() to generate autocomplete suggestions (e.g. IPython kernel, Databricks Notebooks) will suggest column names on the completion df[|]

      However, for those using IPython, they don't check keys, or say it only support keys for pandas & numpy & dict type. So we need to add ipython_key_completions to our object to make it work.

       

      Attachments

        Activity

          People

            jarviscao Beishao Cao
            jarviscao Beishao Cao
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: