[ARROW-9969] [C++] RecordBatchBuilder yields invalid result with dictionary fields

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.1
• Fix Version/s: 2.0.0
• Component/s: C++
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/17342
• Assignee: Troels Nielsen
• Reporter: Pierre Belzile
• Created: 10/Sep/20 17:37
• Updated: 11/Jan/23 08:10
• Resolved: 21/Sep/20 13:00

Description

The record batch builder takes a schema as input and uses that schema when creating the record batch.

However when one or more fields are dictionaries, the data type is unknown until the dictionary builder flushes and the initial schema often does not match. The builder needs to modify the schema for the actual data type generated.

This problem is easily reproduced by providing a schema with a field dictionary(int16(), utf8()) and adding a single row. This yields a data type of dictionary(int8(),utf8()).

Issue Links

• links to: GitHub Pull Request #8178