[ARROW-13625] [C++][CSV] Timestamp parsing should accept any valid ISO 8601 without requiring custom parse strings

• Type: Improvement
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 8.0.0
• Component/s: C++
• External issue URL: https://github.com/apache/arrow/issues/18786
• Assignee: Unassigned
• Reporter: Neal Richardson
• Created: 13/Aug/21 14:56
• Updated: 11/Jan/23 08:34
• Resolved: 13/May/22 18:46

Description

I was trying to read in some git logs and got this parse error for a column I had declared as timestamp type:

Error: Invalid: In CSV column #0: CSV conversion error to timestamp[s]: invalid value '2021-08-11T17:39:50-04:00'

This is valid ISO 8601 and is what git log produces with the I "strict ISO 8601 format" option (https://git-scm.com/docs/pretty-formats).

I see mentioned on ARROW-10343 that timezone indicators are not supported--is that still true? And I recognize that it's not trivial because a timestamp array has to have the same timezone for all values, so if some rows in this CSV had different timezones listed, we would have to handle that (converting everything to UTC is probably the most useful thing but technically loses information).

Issue Links

• relates to: ARROW-10343 [C++] Unable to parse strings into timestamps
• relates to: ARROW-12820 [C++] Strptime ignores timezone information
• relates to: ARROW-14231 [C++] Can't cast from timestamp to utf8 when saving CSV
• relates to: ARROW-13348 [C++] Allow timestamp parser to parse offset strings