Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-7967

cannot resolve 'count' given input columns when using DataFrame.withColumn

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.3.0
    • Fix Version/s: None
    • Component/s: SQL
    • Environment:

      spark 1.3.0 standalone

      Description

      Code:

      val userDF = app_user_register_log.filter($"add_time" > startDay).filter($"add_time" < endDay)
      .select("id").as("userReg")
      .join(activeDF.as("ad"), $"userReg.id" === $"ad.uid")
      .select("ad.uid","ad.clientVerion","ad.loc","ad.auth_status"
      ,"ad.channel","ad.bd_area","ad.mobile_area","ad.idcard_area")
      .withColumn("count", $"count") // Exception came from this line

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              WillCup Will Chen
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: