Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15763 Über-JIRA: abfs phase II: Hadoop 3.3 features & fixes
  3. HADOOP-16479

ABFS FileStatus.getModificationTime returns localized time instead of UTC

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.2.0
    • Fix Version/s: 3.3.0
    • Component/s: fs/azure
    • Labels:
      None
    • Release Note:
      Hide
      private long parseLastModifiedTime(final String lastModifiedTime)

      Timezone is part of lastModifiedTime String as it's last field. But when parsed it ignores timezone field and assumes JVM timezone.
      Fix: Made timezone field considered in lastModifiedTime when the same is parsed
      Show
      private long parseLastModifiedTime(final String lastModifiedTime) Timezone is part of lastModifiedTime String as it's last field. But when parsed it ignores timezone field and assumes JVM timezone. Fix: Made timezone field considered in lastModifiedTime when the same is parsed

      Description

      As per javadoc, the method FileStatus.getModificationTime() should return the time in UTC, but it returns the time in the JVM timezone.

      The issue origins in AzureBlobFileSystemStore.getFileStatus() itself, since  parseLastModifiedTime() returns a wrong date. I have created a file in Azure Data Lake Gen2 and when I look at  it through the Azure Explorer it shows the correct modification time, but the method returns -2 hours time (I am in CET = UTC+2).

      Azure Explorer last modified time:

      AbfsClient parseLastModifiedTime:

      It shows 15:21 CEST as utcDate, when it should be 15:21 UTC, which results in the 2 hour loss.

      DateFormat.parse uses a localized calendar to parse dates which might be the source of the issue.

        Attachments

        1. image-2019-07-31-18-23-37-349.png
          163 kB
          Joan Sala Reixach
        2. image-2019-07-31-18-21-53-023.png
          34 kB
          Joan Sala Reixach
        3. HADOOP-16479-003.patch
          2 kB
          Bilahari T H
        4. HADOOP-16479-002.patch
          3 kB
          Bilahari T H
        5. HADOOP-16479-001.patch
          3 kB
          Bilahari T H

          Issue Links

            Activity

              People

              • Assignee:
                bilahari.th Bilahari T H
                Reporter:
                jsalareixach Joan Sala Reixach
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: