Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-43889

Add check to see if column name is legal for __dir__

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.5.0
    • 3.5.0
    • PySpark
    • None

    Description

      In SPARK-43270, we add a support for the autocomplete suggest for df.| , which also suggest column_name for dataframe. 

      However, we found out later that some dataframe can have column which has illegal variable name (e.g: name?1, name 1, 2name etc.) These variable name should be filtered out and this will be consistent with pandas behavior now

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            jarviscao Beishao Cao
            jarviscao Beishao Cao
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment