Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-18823

Assignation by column name variable not available or bug?

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Question
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.0.2
    • 2.1.1, 2.2.0
    • SparkR
    • None
    • RStudio Server in EC2 Instances (EMR Service of AWS) Emr 4. Or databricks (community.cloud.databricks.com) .

    Description

      I really don't know if this is a bug or can be done with some function:

      Sometimes is very important to assign something to a column which name has to be access trough a variable. Normally, I have always used it with doble brackets likes this out of SparkR problems:

      1. df could be faithful normal data frame or data table.
      2. accesing by variable name:
        myname = "waiting"
        df[[myname]] <- c(1:nrow(df))
      3. or even column number
        df[[2]] <- df$eruptions

      The error is not caused by the right side of the "<-" operator of assignment. The problem is that I can't assign to a column name using a variable or column number as I do in this examples out of spark. Doesn't matter if I am modifying or creating column. Same problem.

      I have also tried to use this with no results:
      val df2 = withColumn(df,"tmp", df$eruptions)

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            felixcheung Felix Cheung
            masip85 Vicente Masip
            Shivaram Venkataraman Shivaram Venkataraman
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - 24h
                24h
                Remaining:
                Remaining Estimate - 24h
                24h
                Logged:
                Time Spent - Not Specified
                Not Specified

                Slack

                  Issue deployment