[ARROW-17424] [R] Microsecond is not sufficient unit for POSIXct

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 9.0.0
• Component/s: R
• External issue URL: https://github.com/apache/arrow/issues/32693
• Assignee: Unassigned
• Reporter: SHIMA Tatsuya
• Created: 15/Aug/22 17:52
• Updated: 11/Jan/23 11:50

Description

I believe the POSIXct type or R currently corresponds to the Arrow timestamp[us, tz=UTC] type.

lubridate::as_datetime(0) |> arrow::infer_type()
#> Timestamp
#> timestamp[us, tz=UTC]

lubridate::as_datetime("1970-01-01 00:00:00.0000001") |>
  arrow::arrow_table(x = _)
#> Table
#> 1 rows x 1 columns
#> $x <timestamp[us, tz=UTC]>

df_a <- lubridate::as_datetime("1970-01-01 00:00:00.0000001") |>
  arrow::arrow_table(x = _) |>
  as.data.frame()

df_b <- lubridate::as_datetime("1970-01-01 00:00:00.0000001") |>
  tibble::tibble(x = _)

waldo::compare(df_a, df_b)
#> `old$x`: "1970-01-01"
#> `new$x`: "1970-01-01 00:00:00"

However, as shown below, POSIXct may hold data finer than a microsecond.

lubridate::as_datetime(0.000000001) |> as.numeric()
#> [1] 1e-09
lubridate::as_datetime("1970-01-01 00:00:00.0000001") |> as.numeric()
#> [1] 1.192093e-07

I don't know why it is currently set in microseconds, but is there any reason not to set it in nanoseconds?

Issue Links

• is related to: ARROW-17425 [R] `lubridate::as_datetime()` in dplyr query should be able to handle time in sub seconds
• relates to: ARROW-15599 [R] Convert a column as a sub-second timestamp from CSV file with the `T` col type option