Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-11211

[R] ChunkedArray$create assumes all chunks are the same type

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: R
    • Labels:
      None

      Description

      It detects the type from the first chunk and uses it for all chunks. Normally this works ok, but it can lead to unexpected behavior, such as:

      data <- list(1:10, NaN)
      x <- chunked_array(!!!data)
      

      returns:
      Error: Invalid: Value is too large to fit in C integer type

      There are a few things that might fix/change this:

      • improved error message
      • chunked arrays not assuming the first chunk's types can be cast safely to all others

      Note that in this case, specifying the type to int64() does "work" with an overflowed NaN value (-9223372036854775808)

      data <- list(1:10, NaN)
      x <- chunked_array(!!!data, type = int64())
      

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              jonkeane Jonathan Keane
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: