[HADOOP-16085] S3Guard: use object version or etags to protect against inconsistent read after replace/overwrite

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.2.0
• Fix Version/s: 3.3.0
• Component/s: fs/s3

Release Note

S3Guard will now track the etag of uploaded files and, if an S3 bucket is versioned, the object version. You can then control how to react to a mismatch between the data in the DynamoDB table and that in the store: warn, fail, or, when using versions, return the original value.

This adds two new columns to the table: etag and version. This is transparent to older S3A clients -but when such clients add/update data to the S3Guard table, they will not add these values. As a result, the etag/version checks will not work with files uploaded by older clients.

For a consistent experience, upgrade all clients to use the latest hadoop version.

• Assignee: Ben Roling
• Reporter: Ben Roling
• Created: 29/Jan/19 15:00
• Updated: 05/Jan/22 16:58
• Resolved: 19/May/19 21:39

Description

Currently S3Guard doesn't track S3 object versions.  If a file is written in S3A with S3Guard and then subsequently overwritten, there is no protection against the next reader seeing the old version of the file instead of the new one.

It seems like the S3Guard metadata could track the S3 object version.  When a file is created or updated, the object version could be written to the S3Guard metadata.  When a file is read, the read out of S3 could be performed by object version, ensuring the correct version is retrieved.

I don't have a lot of direct experience with this yet, but this is my impression from looking through the code.  My organization is looking to shift some datasets stored in HDFS over to S3 and is concerned about this potential issue as there are some cases in our codebase that would do an overwrite.

I imagine this idea may have been considered before but I couldn't quite track down any JIRAs discussing it.  If there is one, feel free to close this with a reference to it.

Am I understanding things correctly?  Is this idea feasible?  Any feedback that could be provided would be appreciated.  We may consider crafting a patch.

Issue Links

• causes: HADOOP-16332 Remove S3A's depedency on http core
• contains: HADOOP-16370 S3AFileSystem copyFile to propagate etag/version from getObjectMetadata to copy request
• depends upon: HADOOP-16190 S3A copyFile operation to include source versionID or etag in the copy request
• is depended upon by: HADOOP-14936 S3Guard: remove "experimental" from documentation
• is related to: HADOOP-16313 multipart/huge file upload tests to look at checksums returned
• is related to: HADOOP-15625 S3A input stream to use etags/version number to detect changed source files
• is related to: HADOOP-15894 getFileChecksum() needs to adopt S3Guard
• is related to: HADOOP-16090 S3A Client to add explicit support for versioned stores
• is related to: HADOOP-16368 S3A list operation doesn't pick up etags from results
• supercedes: HADOOP-16190 S3A copyFile operation to include source versionID or etag in the copy request
• links to: GitHub Pull Request #646
• links to: GitHub Pull Request #675
• links to: GitHub Pull Request #794
• links to: GitHub Pull Request #803
• links to: GitHub Pull Request #807
• links to: GitHub Pull Request #824