[ARROW-13441] [CSV] Streaming reader conversion should skip empty blocks

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 5.0.0
• Fix Version/s: 6.0.0
• Component/s: C++
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/29108
• Assignee: Nate Clark
• Reporter: Nate Clark
• Created: 23/Jul/21 17:05
• Updated: 11/Jan/23 08:33
• Resolved: 31/Aug/21 17:29

Description

The csv streaming reader hardens the schema after the first block is processed. However if the first block does not have any rows then the schema will be hardened with all columns as NAType. This is made worse with the skip_rows_after_names options which will create empty batches until the specified number of rows are skipped.

Issue Links

• links to: GitHub Pull Request #10794