[SPARK-18823] Assignation by column name variable not available or bug?

• Type: Question
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.2
• Fix Version/s: 2.1.1, 2.2.0
• Component/s: SparkR

Environment

RStudio Server in EC2 Instances (EMR Service of AWS) Emr 4. Or databricks (community.cloud.databricks.com) .

• Target Version/s: 2.1.1, 2.2.0
• Assignee: Felix Cheung
• Reporter: Vicente Masip
• Shepherd: Shivaram Venkataraman
• Created: 11/Dec/16 23:45
• Updated: 24/Jan/17 08:24
• Resolved: 24/Jan/17 08:24

Description

I really don't know if this is a bug or can be done with some function:

Sometimes is very important to assign something to a column which name has to be access trough a variable. Normally, I have always used it with doble brackets likes this out of SparkR problems:

1. df could be faithful normal data frame or data table.
2. accesing by variable name:
   myname = "waiting"
   df[[myname]] <- c(1:nrow(df))
3. or even column number
   df[[2]] <- df$eruptions

The error is not caused by the right side of the "<-" operator of assignment. The problem is that I can't assign to a column name using a variable or column number as I do in this examples out of spark. Doesn't matter if I am modifying or creating column. Same problem.

I have also tried to use this with no results:
val df2 = withColumn(df,"tmp", df$eruptions)

Issue Links

• relates to: SPARK-19130 SparkR should support setting and adding new column with singular value implicitly
• links to: [Github] Pull Request #16663 (felixcheung)