[ARROW-10114] [R] Segfault in to_dataframe_parallel with deeply nested structs

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.1
• Fix Version/s: 2.0.0
• Component/s: R
• Labels: pull-request-available

Environment

> sessionInfo()
R version 3.6.3 (2020-02-29)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Linux Mint 19.3

Matrix products: default
BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.7.1
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.7.1

locale:
 [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
 [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
 [5] LC_MONETARY=sv_SE.UTF-8 LC_MESSAGES=en_US.UTF-8
 [7] LC_PAPER=sv_SE.UTF-8 LC_NAME=C
 [9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=sv_SE.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] arrow_1.0.1

loaded via a namespace (and not attached):
 [1] tidyselect_1.1.0 bit_4.0.4 compiler_3.6.3 magrittr_1.5
 [5] assertthat_0.2.1 R6_2.4.1 glue_1.4.1 Rcpp_1.0.5
 [9] bit64_4.0.2 vctrs_0.3.2 rlang_0.4.7 purrr_0.3.4

• External issue URL: https://github.com/apache/arrow/issues/26127
• Assignee: Romain Francois
• Reporter: Markus Skyttner
• Created: 28/Sep/20 10:52
• Updated: 11/Jan/23 08:11
• Resolved: 09/Oct/20 22:26

Description

A .jsonl file (newline separated JSON) created from open data available at ftp://ftp.libris.kb.se/pub/spa/swepub-deduplicated-2019-12-29.zip is used with the R package arrow (installed from CRAN) using the following statement:

> arrow::read_json_arrow("~/.config/swepub/head.jsonl")

It crashes RStudio with no error message. At the R prompt, the error message is:

Error in Table__to_dataframe(x, use_threads = option_use_threads()) :
SET_VECTOR_ELT() can only be applied to a 'list', not a 'integer'

The file "head.jsonl" above was created from the extracted zip's .jsonl file with the *nix "head -1 $BIG_JSONL_FILE" command. It can be parsed with jsonlite and tidyjson.

Also got this error message at one point:

> arrow::read_json_arrow("head.jsonl", as_data_frame = TRUE)

• • • caught segfault ***
      address 0x8, cause 'memory not mapped'

Traceback:
1: structure(x, extra_cols = colonnade[extra_cols], class = "pillar_squeezed_colonnade")
2: new_colonnade_sqeezed(out, colonnade = x, extra_cols = extra_cols)
3: pillar::squeeze(x$mcf, width = width)
4: format.trunc_mat(mat)
5: format(mat)
6: format.tbl(x, ..., n = n, width = width, n_extra = n_extra)
7: format(x, ..., n = n, width = width, n_extra = n_extra)
8: paste0(..., collapse = "\n")
9: cli::cat_line(format(x, ..., n = n, width = width, n_extra = n_extra))
10: print.tbl
11: (function (x, ...) UseMethod("print"))

Issue Links

• fixes: ARROW-9676 [R] Error converting Table with nested structs
• relates to: ARROW-8379 [R] Investigate/fix thread safety issues (esp. Windows)
• links to: GitHub Pull Request #8411