Details
-
Improvement
-
Status: Resolved
-
P2
-
Resolution: Fixed
-
2.5.0
-
None
Description
moveToOutput() methods in FileBasedSink.WriteOperation implements move by copy+delete. It would be better to use a rename() which can be much more effective for some filesystems.
Filesystem must support cross-directory rename. BEAM-4861 is related to this for the case of HDFS filesystem.
Feature was discussed here: