When using S3Guard in authoritative mode a file can be reported from AWS S3 that's missing like it is described in the following exception:
java.io.FileNotFoundException: re-open s3a://cloudera-dev-gabor-ireland/test/TMCDOR-021df1ad-633f-47b8-97f5-6cd93f0b82d0 at 0 on
com.amazonaws.services.s3.model.AmazonS3Exception: The specified key does not exist. (Service: Amazon S3; Status Code: 404; Error
Code: NoSuchKey; Request ID: E1FF9EA9B5DBBD7E; S3 Extended Request ID:
NzNIL4+dyA89WTnfbcwuYQK+hCfx51TfavwgC3oEvQI0IQ9M/zAspbXOfBIis8/nTolc4tRB9ik=), S3 Extended Request ID:
But the metadata in S3Guard (e.g dynamo db) is there, so it can be opened. The operation will not fail when it's opened, it will fail when we try to read it, so the call
FSDataInputStream is = guardedFs.open(testFilePath);
won't fail, but the next call
byte firstRead = new byte[text.length()];
is.read(firstRead, 0, firstRead.length);
will fail with the exception message like what's above.
Once Authoritative mode is on, we assume that there is no out of band operation, so the file will appear eventually. We should re-try in this case.