[ARROW-15599] [R] Convert a column as a sub-second timestamp from CSV file with the `T` col type option

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 6.0.1
• Fix Version/s: 8.0.0
• Component/s: R
• Labels: pull-request-available
• Environment: R version 4.1.2 (2021-11-01) ; Platform: x86_64-pc-linux-gnu (64-bit) ; Running under: Ubuntu 20.04.3 LTS
• External issue URL: https://github.com/apache/arrow/issues/31061
• Assignee: SHIMA Tatsuya
• Reporter: SHIMA Tatsuya
• Created: 07/Feb/22 13:45
• Updated: 11/Jan/23 11:38
• Resolved: 02/Mar/22 14:02

Description

I tried to read the csv column type as timestamp, but I could only get it to work well when `col_types` was not specified.

I'm sorry if I missed something and this is the expected behavior. (It would be great if you could add an example with `col_types` in the documentation.)

library(arrow)
#>
#> Attaching package: 'arrow'
#> The following object is masked from 'package:utils':
#>
#>     timestamp

t_string <- tibble::tibble(
  x = "2018-10-07 19:04:05.005"
)

write_csv_arrow(t_string, "tmp.csv")

read_csv_arrow(
  "tmp.csv",
  as_data_frame = FALSE
)
#> Table
#> 1 rows x 1 columns
#> $x <timestamp[ns]>

read_csv_arrow(
  "tmp.csv",
  col_names = "x",
  col_types = "?",
  skip = 1,
  as_data_frame = FALSE
)
#> Table
#> 1 rows x 1 columns
#> $x <timestamp[ns]>

read_csv_arrow(
  "tmp.csv",
  col_names = "x",
  col_types = "T",
  skip = 1,
  as_data_frame = FALSE
)
#> Error: Invalid: In CSV column #0: CSV conversion error to timestamp[s]: invalid value '2018-10-07 19:04:05.005'

read_csv_arrow(
  "tmp.csv",
  col_names = "x",
  col_types = "T",
  as_data_frame = FALSE,
  skip = 1,
  timestamp_parsers = "%Y-%m-%d %H:%M:%S"
)
#> Error: Invalid: In CSV column #0: CSV conversion error to timestamp[s]: invalid value '2018-10-07 19:04:05.005'

Issue Links

• is related to: ARROW-17424 [R] Microsecond is not sufficient unit for POSIXct
• is related to: ARROW-17425 [R] `lubridate::as_datetime()` in dplyr query should be able to handle time in sub seconds
• relates to: ARROW-17429 [R] Error messages are not helpful of read_csv_arrow with col_types option
• links to: GitHub Pull Request #12474