[CARBONDATA-2745] Add a seperate Impl for AtomicFileOperations for s3

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.4.1
• Assignee: Kunal Kapoor
• Reporter: Kunal Kapoor
• Created: 16/Jul/18 09:00
• Updated: 24/Jul/18 13:11
• Resolved: 24/Jul/18 13:10

Description

AtomicFileOperationImpl creates a temporary file and then renames the file to actual file name. This is risky in S3 storage as the file has to be deleted and then recreated. 

 

S3 supports atomic file overwrite. hdfs rename is atomic, while overwrite is not atomic and can result in empty file read temporarily.

So separate implementations for both hdfs and S3 to ensure consistancy of overwrite and read

Issue Links

• links to: GitHub Pull Request #2511