Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-16829 Über-jira: S3A Hadoop 3.4 features
  3. HADOOP-16382

Clock skew can cause S3Guard to think object metadata is out of date

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 3.3.0
    • Fix Version/s: None
    • Component/s: fs/s3
    • Labels:
      None

      Description

      When a S3Guard entry is added for an object, its last updated flag is taken from the local clock: if a getFileStatus is made immediately afterwards, the timestamp of the file from the HEAD may be > than the local time, so the DDB entry updated.

      This is even if the clocks are close. When updating an entry from S3, the actual timestamp of the file should be used to fix it, not local clocks

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                stevel@apache.org Steve Loughran
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: