Description
If an S3A rename() operation fails partway through, such as when the user doesn't have permissions to delete the source files after copying to the destination, then the s3guard view of the world ends up inconsistent. In particular the sequence
(assuming src/file* is a list of files file1...file10 and read only to caller)
- create file rename src/file1 dest/ ; expect AccessDeniedException in the delete, dest/file1 will exist
- delete file dest/file1
- rename src/file* dest/ ; expect failure
- list dest; you will not see dest/file1
You will not see file1 in the listing, presumably because it will have a tombstone marker and the update at the end of the rename() didn't take place: the old data is still there.
Attachments
Attachments
Issue Links
- causes
-
HADOOP-16390 Build fails due to bad use of '>' in javadoc
- Resolved
- contains
-
HADOOP-15604 Bulk commits of S3A MPUs place needless excessive load on S3 & S3Guard
- Resolved
-
HADOOP-15658 Memory leak in S3AOutputStream
- Resolved
-
HADOOP-16364 S3Guard table destroy to map IllegalArgumentExceptions to IOEs
- Resolved
- depends upon
-
HADOOP-15176 Enhance IAM Assumed Role support in S3A client
- Resolved
- incorporates
-
HADOOP-13936 S3Guard: DynamoDB can go out of sync with S3AFileSystem.delete()
- Resolved
- is depended upon by
-
HADOOP-14936 S3Guard: remove "experimental" from documentation
- Resolved
-
HADOOP-15492 increase performance of s3guard import command
- Resolved
- is related to
-
HADOOP-16380 S3A tombstones can confuse empty directory status
- Resolved
-
HADOOP-15621 S3Guard: Implement time-based (TTL) expiry for Authoritative Directory Listing
- Resolved
-
HADOOP-16430 S3AFilesystem.delete to incrementally update s3guard with deletions
- Resolved
-
HADOOP-17869 fs.s3a.connection.maximum should be bigger than fs.s3a.threads.max
- Resolved
- links to