[SPARK-28040] sql() fails to process output of glue::glue_data() - ASF JIRA

Attach files

Attach Screenshot

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Bug
Status: Resolved
Priority: Major
Resolution: Incomplete
Affects Version/s: 2.4.3
Fix Version/s: None
Component/s: SparkR
Labels:
- bulk-closed

Description

glue package is quite natural for sending parameterized queries to Spark from R. Very similar to Python's format for strings. Error is as simple as

library(glue)
library(sparkR)
sparkR.session()

query = glue_data(list(val = 4), 'select {val}')
sql(query)

Error in writeType(con, serdeType) :
Unsupported type for serialization glue
sql(as.character(query)) works as expected but this is a bit awkward / post-hoc