Details
-
Sub-task
-
Status: Closed
-
Major
-
Resolution: Won't Do
-
None
-
None
-
None
Description
The dataset writer needs an LRU cache to keep track of open files so that it can respect a "max open files" property (see ARROW-12321). A synchronous LruCache implementation already exists but on eviction from the cache we need to wait until all pending writes have completed before we evict the item and open a new file. This ticket is to create an AsyncLruCache which will allow the creation of items and the eviction of items to be asynchronous.