[ARROW-2457] garrow_array_builder_append_values() won't work for large arrays

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.8.0, 0.9.0
• Fix Version/s: 0.10.0
• Component/s: C++, GLib
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/18613
• Assignee: Kouhei Sutou
• Reporter: Haralampos Gavriilidis
• Created: 13/Apr/18 15:24
• Updated: 11/Jan/23 07:21
• Resolved: 21/Apr/18 13:51

Description

I am using garrow_array_builder_append_values() to transform a native C array to an Arrow array, without calling arrow_array_builder_append multiple times. When calling garrow_array_builder_append_values() in array-builder.cpp with following signature:

garrow_array_builder_append_values(GArrowArrayBuilder *builder,
const VALUE *values,
gint64 values_length,
const gboolean *is_valids,
gint64 is_valids_length,
GError **error,
const gchar *context)

it will fail for large arrays. This is probably happening because the is_valids array is copied to the valid_bytes array (of different type), for which the memory is allocated on the stack, and not on the heap, like shown on the snippet below:

uint8_t valid_bytes[is_valids_length];
for (gint64 i = 0; i < is_valids_length; ++i){ 
  valid_bytes[i] = is_valids[i]; 
}

 A way to avoid this problem would be to allocate memory for the valid_bytes array using malloc() or something similar. Is this behavior intended, maybe because no large arrays should be handed over to that function, or it is rather a bug?

Issue Links

• links to: GitHub Pull Request #1921