[ARROW-6775] [C++] [Python] Proposal for several Array utility functions

• Type: Wish
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 1.0.0
• Component/s: C++, Python
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/16913
• Assignee: Wes McKinney
• Reporter: Zhuo Peng
• Created: 02/Oct/19 22:42
• Updated: 11/Jan/23 07:49
• Resolved: 07/Jul/20 11:23

Description

Hi,

We developed several utilities that computes / accesses certain properties of Arrays and wonder if they make sense to get them into the upstream (into both the C++ API and pyarrow) and assuming yes, where is the best place to put them?

Maybe I have overlooked existing APIs that already do the same.. in that case please point out.

 

1/ ListLengthFromListArray(ListArray&)

Returns lengths of lists in a ListArray, as a Int32Array (or Int64Array for large lists). For example:

[[1, 2, 3], [], None] => [3, 0, 0] (or [3, 0, None], but we hope the returned array can be converted to numpy)

 

2/ GetBinaryArrayTotalByteSize(BinaryArray&)

Returns the total byte size of a BinaryArray (basically offset[len - 1] - offset[0]).

Alternatively, a BinaryArray::Flatten() -> Uint8Array would work.

 

3/ GetArrayNullBitmapAsByteArray(Array&)

Returns the array's null bitmap as a UInt8Array (which can be efficiently converted to a bool numpy array)

 

4/ GetFlattenedArrayParentIndices(ListArray&)

Makes a int32 array of the same length as the flattened ListArray. returned_array[i] == j means i-th element in the flattened ListArray came from j-th list in the ListArray.

For example [[1,2,3], [], None, [4,5]] => [0, 0, 0, 3, 3]

 

Issue Links

• relates to: ARROW-8894 [C++] C++ array kernels framework and execution buildout (umbrella issue)
• relates to: ARROW-9248 [C++] Add "list_size" function that returns Int32Array/Int64Array giving list cell sizes
• relates to: ARROW-9249 [C++] Implement "list_parent_indices" vector function
• relates to: ARROW-9247 [Python] Expose BinaryArray::total_values_length in bindings
• relates to: ARROW-9116 [C++] Add BinaryArray::total_values_length()
• links to: GitHub Pull Request #7632