[ARROW-2380] [Python] Correct issues in numpy_to_arrow conversion routines

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.9.0
• Fix Version/s: 0.10.0
• Component/s: Python
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/18404
• Assignee: Antoine Pitrou
• Reporter: Bryan Cutler
• Created: 02/Apr/18 21:16
• Updated: 11/Jan/23 07:20
• Resolved: 05/Apr/18 13:35

Description

Following the discussion at https://github.com/apache/arrow/pull/1689, there are a few issues with conversion of various types to arrow that are incorrect or could be improved:

• PyBytes_GET_SIZE is being casted to the wrong type, for example
  const int32_t length = static_cast<int32_t>(PyBytes_GET_SIZE(obj));

• Handle the possibility with the statement
  builder->value_data_length() + length > kBinaryMemoryLimit
  if length is larger than kBinaryMemoryLimit

• Look into using common code for binary object conversion to avoid duplication, and allow support for bytes and bytearray objects in other places than numpy_to_arrow. (possibly put in src/arrow/python/helpers.h)

Issue Links

• links to: GitHub Pull Request #1835