Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
Impala 3.3.0
-
None
-
None
-
ghx-label-5
Description
Currently, Impala will delete old data cache files upon restart but it only does so when data cache is enabled. However, if the user turns off the data cache after restart, the old data cache files may be left hanging around until the next restart with the cache enabled. We should have an option to delete the data cache files on Impala shutdown. The initial implementation of the data cache has the unlink-on-create behavior but it was confusing to users as there will be unaccounted usage on the storage side (as the file is not easily visible after unlink). We may want to consider support this "unlink-on-create" behavior behind a flag in case users prefer to have the data cache files removed upon Impala's exit. In the long run, we probably want to have an "exit handler" in Impala to make sure resources are freed up on Impala exit.