[HADOOP-14585] Ensure controls in-place to prevent clients with significant clock skews pruning aggressively

• Type: Sub-task
• Status: Resolved
• Priority: Minor
• Resolution: Won't Fix
• Affects Version/s: 3.0.0-beta1
• Component/s: fs/s3
• Assignee: Unassigned
• Reporter: Sean Mackrory
• Created: 23/Jun/17 23:41
• Updated: 27/Feb/21 11:00
• Resolved: 27/Feb/21 11:00

Description

From discussion on HADOOP-14499:

bear in mind that we can't guarantee that the clocks of all clients are in sync; you don't want a client whose TZ setting is wrong to aggressively prune things. Had that happen in production with files in shared filestore. This is why ant -diagnostics checks time consistency with temp files...

temp files work on a shared FS. AWS is actually somewhat sensitive to clocks: if your VM is too far out of time then auth actually fails, its ~+-15 minutes. There's some stuff in the Java SDK to actually calculate and adjust clock skew, presumably parsing the timestamp of a failure, calculating the difference and retrying. Which means that the field in SDKGlobalConfiguration could help identify the difference between local time and AWS time.