[ARROW-3780] [R] Failed to fetch data: invalid data when collecting int16

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.14.0
• Component/s: R
• Labels: pull-request-available spark
• External issue URL: https://github.com/apache/arrow/issues/15870
• Assignee: Romain Francois
• Reporter: Javier Luraschi
• Created: 13/Nov/18 19:58
• Updated: 11/Jan/23 07:29
• Resolved: 12/Jun/19 11:37

Description

Repro from sparklyr unit test:

library(dplyr)
library(sparklyr)
library(arrow)

sc <- spark_connect(master = "local")

hive_type <- tibble::frame_data(
 ~stype, ~svalue, ~rtype, ~rvalue, ~arrow,
 "smallint", "1", "integer", "1", "integer",
)

spark_query <- hive_type %>%
 mutate(
 query = paste0("cast(", svalue, " as ", stype, ") as ", gsub("\\(|\\)", "", stype), "_col")
 ) %>%
 pull(query) %>%
 paste(collapse = ", ") %>%
 paste("SELECT", .)

spark_types <- DBI::dbGetQuery(sc, spark_query) %>%
 lapply(function(e) class(e)[[1]]) %>%
 as.character()

Actual: error: Failed to fetch data: invalid data 

Issue Links

• links to: GitHub Pull Request #2953
• links to: GitHub Pull Request #4505
• links to: GitHub Pull Request #4529