[ARROW-16421] [R] Permission error on Windows when deleting file previously accessed with open_dataset

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 7.0.0
• Component/s: R
• External issue URL: https://github.com/apache/arrow/issues/31796
• Assignee: Unassigned
• Reporter: Will Jones
• Created: 29/Apr/22 18:24
• Updated: 11/Jan/23 11:43

Description

On Windows this fails:

library(arrow)

write_dataset(iris, "test_dataset")

# Original example was with DuckDB, but that's not necessarily the issue
# con <- open_dataset("test_dataset") |> to_duckdb()
con <- open_dataset("test_dataset")$NewScan()$Finish()$ToRecordBatchReader()

file.remove("test_dataset/part-0.parquet")
#> Warning in file.remove("test_dataset/part-0.parquet"): cannot remove file
#> 'test_dataset/part-0.parquet', reason 'Permission denied'
#> [1] FALSE

But on MacOS it does not:

library(arrow)

write_dataset(iris, "test_dataset")

# Original example was with DuckDB, but that's not necessarily the issue
# con <- open_dataset("test_dataset") |> to_duckdb()
con <- open_dataset("test_dataset")$NewScan()$Finish()$ToRecordBatchReader()

file.remove("test_dataset/part-0.parquet")
#> [1] TRUE

Issue Links

• is duplicated by: ARROW-17002 [R] dplyr queries create locks on FileSystemDataset files
• is related to: ARROW-13763 [Python] Files opened for read with pyarrow.parquet are not explicitly closed
• supercedes: ARROW-17444 [R] Windows Only: Cannot delete file previously accesed with open_dataset