[ARROW-9676] [R] Error converting Table with nested structs

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.0
• Fix Version/s: 2.0.0
• Component/s: R
• Environment: Amazon Linux, 32gb of ram
• External issue URL: https://github.com/apache/arrow/issues/17303
• Assignee: Romain Francois
• Reporter: Nick DiQuattro
• Created: 09/Aug/20 00:47
• Updated: 11/Jan/23 08:08
• Resolved: 01/Nov/20 22:51

Description

When trying to collect data from a dataset based on parquet files with nested structs (column is a struct with 2 structs nested) of moderate size (1Mish rows), R crashes. If I add a filter to reduce the number of rows, the data is parsed. If I select out the struct column, it works great (up to 21M rows). My hunch is the structs resulting in data.frame columns may be the issue. I am curious if there's a way to have arrow import structs as lists instead of data.frames. Thanks for the direction to here neilr8133!

Issue Links

• is fixed by: ARROW-10114 [R] Segfault in to_dataframe_parallel with deeply nested structs