Details
-
Sub-task
-
Status: In Progress
-
Major
-
Resolution: Unresolved
-
None
-
None
Description
From a discussion on a PR which documents the encoding argument (https://github.com/apache/arrow/pull/13038)
Currently if we want to specify Arrow-specific read options such as encoding, we'd have to do something like this:
df <- read_csv_arrow(tf, read_options = CsvReadOptions$create(encoding = "utf8"))
However, this uses a lower-level API that we don't want to include in the examples for end-users to see.
We should update the code inside read_csv_arrow() so that the user can specify read_options as a list which we then pass through to CsvReadOptions internally, so we could instead call the much more user-friendly code below:
df <- read_csv_arrow(tf, read_options = list(encoding = "utf8"))
We should then add an example of this to the function doc examples.
We also should do the same for parse_options and convert_options.
Similarly, we can do:
open_dataset("data.csv", format = "csv", convert_options = CsvConvertOptions$create(null_values = "Not Range", strings_can_be_null = TRUE))%>% collect()
but it'd be great to be able to do:
open_dataset("data.csv", format = "csv", convert_options = list(null_values = "Not Range", strings_can_be_null = TRUE))%>% collect()
Attachments
Issue Links
- links to
