Details
Major Description
When specifying two columns within an `orderBy()` function, to attempt to get an ordering by two columns in descending order, an error is returned.
library(magrittr) library(SparkR) cars <- cbind(model = rownames(mtcars), mtcars) carsDF <- createDataFrame(cars) carsDF %>% mutate(rank = over(rank(), orderBy(windowPartitionBy(column("cyl")), desc(column("mpg")), desc(column("disp"))))) %>% head()
This returns an error:
Error in ns[[i]] : subscript out of bounds
This seems to be related to the more general issue that the following code, excluding the use of the `desc()` function also fails:
carsDF %>% mutate(rank = over(rank(), orderBy(windowPartitionBy(column("cyl")), column("mpg"), column("disp")))) %>% head()
Attachments
Attachments
Issue Links
- duplicates
-
SPARK-26199 Long expressions cause mutate to fail
-
- Resolved
-
- is duplicated by
-
-
- Resolved
-
- links to
