[ARROW-11211] [R] ChunkedArray$create assumes all chunks are the same type

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Component/s: R
• Assignee: Unassigned
• Reporter: Jonathan Keane
• Created: 11/Jan/21 20:14
• Updated: 11/Jan/21 21:13

Description

It detects the type from the first chunk and uses it for all chunks. Normally this works ok, but it can lead to unexpected behavior, such as:

data <- list(1:10, NaN)
x <- chunked_array(!!!data)

returns:
Error: Invalid: Value is too large to fit in C integer type

There are a few things that might fix/change this:

• improved error message
• chunked arrays not assuming the first chunk's types can be cast safely to all others

Note that in this case, specifying the type to int64() does "work" with an overflowed NaN value (-9223372036854775808)

data <- list(1:10, NaN)
x <- chunked_array(!!!data, type = int64())