[SPARK-40087] Support multiple Column drop in R

• Type: New Feature
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.3.0
• Fix Version/s: 3.4.0
• Component/s: R
• Docs Text: Support for SparkR to drop multiple "Column"
• Assignee: Santosh Pingale
• Reporter: Santosh Pingale
• Created: 15/Aug/22 17:19
• Updated: 12/Dec/22 18:10
• Resolved: 18/Aug/22 13:03

Description

This is a followup on SPARK-39895. The PR previously attempted to adjust implementation for R as well to match signatures but that part was removed and we only focused on getting python implementation to behave correctly.

Change supports following operations:

df <- select(read.json(jsonPath), "name", "age")

df$age2 <- df$age

df1 <- drop(df, df$age, df$name)
expect_equal(columns(df1), c("age2"))

df1 <- drop(df, df$age, column("random"))
expect_equal(columns(df1), c("name", "age2"))

df1 <- drop(df, df$age, df$name)
expect_equal(columns(df1), c("age2"))

 

 

 

Issue Links

• is related to: SPARK-39895 pyspark drop doesn't accept *cols
• links to: [Github] Pull Request #37526 (santosh-d3vpl3x)