Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13946

Document how HDFS updates timestamps in the FS spec; compare with object stores

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.7.3
    • Fix Version/s: 2.8.0, 3.0.0-alpha4
    • Component/s: documentation, fs
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      SPARK-17159 shows that the behavior of when HDFS updates timestamps isn't well documented. Document these in the FS spec.

      I'm not going to add tests for this, as it is so very dependent on FS implementations, as in "POSIX filesystems may behave differently from HDFS". If someone knows what happens there, their contribution is welcome.

      1. HADOOP-13946-001.patch
        2 kB
        Steve Loughran
      2. HADOOP-13946-002.patch
        4 kB
        Steve Loughran
      3. HADOOP-13946-003.patch
        4 kB
        Steve Loughran
      4. HADOOP-13946-004.patch
        5 kB
        Steve Loughran
      5. HADOOP-13946-005.patch
        5 kB
        Mingliang Liu
      6. Screen Shot 2017-03-07 at 3.44.43 PM.png
        679 kB
        Mingliang Liu

        Issue Links

          Activity

          Hide
          stevel@apache.org Steve Loughran added a comment -

          Patch 001

          Show
          stevel@apache.org Steve Loughran added a comment - Patch 001
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 11s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 mvninstall 12m 36s trunk passed
          +1 mvnsite 0m 57s trunk passed
          +1 mvnsite 0m 51s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 asflicense 0m 15s The patch does not generate ASF License warnings.
          15m 13s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-13946
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12845397/HADOOP-13946-001.patch
          Optional Tests asflicense mvnsite
          uname Linux 2aa1636e3457 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / b31e195
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11343/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 mvninstall 12m 36s trunk passed +1 mvnsite 0m 57s trunk passed +1 mvnsite 0m 51s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 15m 13s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13946 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12845397/HADOOP-13946-001.patch Optional Tests asflicense mvnsite uname Linux 2aa1636e3457 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / b31e195 modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11343/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          liuml07 Mingliang Liu added a comment -

          +1

          Show
          liuml07 Mingliang Liu added a comment - +1
          Hide
          liuml07 Mingliang Liu added a comment -

          Committed to trunk through branch-2.8 branches. Thanks for the contribution Steve Loughran.

          One trivial changes was that I replaced the two "+" with "*" for list chars following convention in the markdown file.

          Show
          liuml07 Mingliang Liu added a comment - Committed to trunk through branch-2.8 branches. Thanks for the contribution Steve Loughran . One trivial changes was that I replaced the two "+" with "*" for list chars following convention in the markdown file.
          Hide
          cnauroth Chris Nauroth added a comment -

          Reopened just to change resolution from Cannot Reproduce to Fixed.

          Show
          cnauroth Chris Nauroth added a comment - Reopened just to change resolution from Cannot Reproduce to Fixed.
          Hide
          liuml07 Mingliang Liu added a comment -

          Thanks Chris Nauroth for taking care of this. I didn't mean it but my mouse must have fooled me.

          Show
          liuml07 Mingliang Liu added a comment - Thanks Chris Nauroth for taking care of this. I didn't mean it but my mouse must have fooled me.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11065 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11065/)
          HADOOP-13946. Document how HDFS updates timestamps in the FS spec; (liuml07: rev 451efb08fe0680d002c6856c104ebb366acee8a0)

          • (edit) hadoop-common-project/hadoop-common/src/site/markdown/filesystem/introduction.md
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11065 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11065/ ) HADOOP-13946 . Document how HDFS updates timestamps in the FS spec; (liuml07: rev 451efb08fe0680d002c6856c104ebb366acee8a0) (edit) hadoop-common-project/hadoop-common/src/site/markdown/filesystem/introduction.md
          Hide
          cnauroth Chris Nauroth added a comment -

          Sorry to come late to the review, but I would have liked to see a mention of how HDFS rename updates the modification time of both the source and the destination folder (though not the modification time of the renamed file itself).

          Also, regarding this:

          Object stores have a significantly simpler view of time:
          ...
          + A file's modification time is always the same as its creation time.

          This makes it sound like this section covers all object stores, but the statement about modification time is not necessarily true universally. For example, on WASB, the FileStatus on read is always populated with the last modified time field of the blob as reported by the Azure Storage service. I think any kind of modification of the blob will result in a change in that value. I specifically tested hadoop fs -chmod against WASB, and it updated the blob's modification time, which is different from HDFS. Out-of-band blob modifications directly through the Azure Storage service, bypassing the FileSystem API, could be another source of perceived changes in the last modification time.

          I expect this is not consistent across services, and therefore it's unlikely we can make accurate statements in the file system spec beyond just saying "it's different."

          Please feel free to address this either by reverting and revising or filing a new JIRA to track an addendum.

          Thanks!

          Show
          cnauroth Chris Nauroth added a comment - Sorry to come late to the review, but I would have liked to see a mention of how HDFS rename updates the modification time of both the source and the destination folder (though not the modification time of the renamed file itself). Also, regarding this: Object stores have a significantly simpler view of time: ... + A file's modification time is always the same as its creation time. This makes it sound like this section covers all object stores, but the statement about modification time is not necessarily true universally. For example, on WASB, the FileStatus on read is always populated with the last modified time field of the blob as reported by the Azure Storage service. I think any kind of modification of the blob will result in a change in that value. I specifically tested hadoop fs -chmod against WASB, and it updated the blob's modification time, which is different from HDFS. Out-of-band blob modifications directly through the Azure Storage service, bypassing the FileSystem API, could be another source of perceived changes in the last modification time. I expect this is not consistent across services, and therefore it's unlikely we can make accurate statements in the file system spec beyond just saying "it's different." Please feel free to address this either by reverting and revising or filing a new JIRA to track an addendum. Thanks!
          Hide
          Thomas Demoor Thomas Demoor added a comment -

          Another late review remark:

          The file only becomes visible at the end of the write operation; this also sets the creation time of the file.

          On S3 an object (file) is only visible at the end of the write operation, but the creation / modification timestamp (Last-Modified HTTP header) is that of the start of the write operation.

          Show
          Thomas Demoor Thomas Demoor added a comment - Another late review remark: The file only becomes visible at the end of the write operation; this also sets the creation time of the file. On S3 an object (file) is only visible at the end of the write operation, but the creation / modification timestamp (Last-Modified HTTP header) is that of the start of the write operation.
          Hide
          liuml07 Mingliang Liu added a comment -

          I've reverted for addressing the comments. Thanks,

          Show
          liuml07 Mingliang Liu added a comment - I've reverted for addressing the comments. Thanks,
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11067 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11067/)
          Revert "HADOOP-13946. Document how HDFS updates timestamps in the FS (liuml07: rev 88731c731a7207edbd5d408a8ec67b2bb63a3eb5)

          • (edit) hadoop-common-project/hadoop-common/src/site/markdown/filesystem/introduction.md
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11067 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11067/ ) Revert " HADOOP-13946 . Document how HDFS updates timestamps in the FS (liuml07: rev 88731c731a7207edbd5d408a8ec67b2bb63a3eb5) (edit) hadoop-common-project/hadoop-common/src/site/markdown/filesystem/introduction.md
          Hide
          stevel@apache.org Steve Loughran added a comment -

          I think I've written this badly. I Had in my head write == the create() call, but from the S3 protocol, it's when the PUT request began. Does this hold for multipart too?

          Show
          stevel@apache.org Steve Loughran added a comment - I think I've written this badly. I Had in my head write == the create() call, but from the S3 protocol, it's when the PUT request began. Does this hold for multipart too?
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Chris, this is my transcribed notes from what you told me, so if it's wrong, then there are thnings you forget to tell me. Actually, for Spark I was only worried about timestamps of files, so ignoring that of directories.

          I'll update the doc

          Show
          stevel@apache.org Steve Loughran added a comment - Chris, this is my transcribed notes from what you told me, so if it's wrong, then there are thnings you forget to tell me. Actually, for Spark I was only worried about timestamps of files, so ignoring that of directories. I'll update the doc
          Hide
          cnauroth Chris Nauroth added a comment -

          It's definitely possible that I messed up. I've lost our email thread on this, but as I recall, the point about rename and handling modification time of directories came up in a follow-up discussion, so it would have been easy for that to get lost in the shuffle.

          Thank you for working on the updates.

          Show
          cnauroth Chris Nauroth added a comment - It's definitely possible that I messed up. I've lost our email thread on this, but as I recall, the point about rename and handling modification time of directories came up in a follow-up discussion, so it would have been easy for that to get lost in the shuffle. Thank you for working on the updates.
          Hide
          Thomas Demoor Thomas Demoor added a comment -

          Yup, same for multipart. The timestamp of the initiate multipart upload request is used as creation/modification time.

          Show
          Thomas Demoor Thomas Demoor added a comment - Yup, same for multipart. The timestamp of the initiate multipart upload request is used as creation/modification time.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Patch 002; cover some aspects of HDFS directory mod times, make object store coverage a lot vaguer and full of caveats.

          I think we need a couple of lines of more detail on directories & HDFS.

          Specifically:

          1. creation time? That of mkdir()?
          2. what operations on child entries update the modtime? mkdir, create, delete, rename. And which don't?
          3. chmod and set time calls?
          Show
          stevel@apache.org Steve Loughran added a comment - Patch 002; cover some aspects of HDFS directory mod times, make object store coverage a lot vaguer and full of caveats. I think we need a couple of lines of more detail on directories & HDFS. Specifically: creation time? That of mkdir()? what operations on child entries update the modtime? mkdir, create, delete, rename. And which don't? chmod and set time calls?
          Hide
          stevel@apache.org Steve Loughran added a comment -

          +Thomas: what happens on COPY

          Show
          stevel@apache.org Steve Loughran added a comment - +Thomas: what happens on COPY
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 mvninstall 12m 31s trunk passed
          +1 mvnsite 0m 54s trunk passed
          +1 mvnsite 0m 49s the patch passed
          -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          +1 asflicense 0m 16s The patch does not generate ASF License warnings.
          15m 7s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-13946
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12846225/HADOOP-13946-002.patch
          Optional Tests asflicense mvnsite
          uname Linux 45603f0a39b7 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 679478d
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/11395/artifact/patchprocess/whitespace-eol.txt
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11395/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 mvninstall 12m 31s trunk passed +1 mvnsite 0m 54s trunk passed +1 mvnsite 0m 49s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 asflicense 0m 16s The patch does not generate ASF License warnings. 15m 7s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13946 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12846225/HADOOP-13946-002.patch Optional Tests asflicense mvnsite uname Linux 45603f0a39b7 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 679478d whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/11395/artifact/patchprocess/whitespace-eol.txt modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11395/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          Thomas Demoor Thomas Demoor added a comment -

          A copy is an independent immutable object (which happens to have the same data contents as the source). It's creation time is that of the PUT-COPY request, not that of the "sibling" object.

          Show
          Thomas Demoor Thomas Demoor added a comment - A copy is an independent immutable object (which happens to have the same data contents as the source). It's creation time is that of the PUT-COPY request, not that of the "sibling" object.
          Hide
          cnauroth Chris Nauroth added a comment -

          creation time? That of mkdir()?

          I think I'm confused by the mentions of creation time. We have mtime and atime in FileStatus. AFAIK, the inode data structures in the NameNode don't track a separate notion of creation time, just mtime and atime. Is there something I've missed?

          what operations on child entries update the modtime? mkdir, create, delete, rename. And which don't?

          Yes, and to this please also add concat.

          chmod and set time calls?

          chmod (and setacl) and setTimes do not alter any modification times, neither on the target path itself nor its parent.

          Show
          cnauroth Chris Nauroth added a comment - creation time? That of mkdir()? I think I'm confused by the mentions of creation time. We have mtime and atime in FileStatus . AFAIK, the inode data structures in the NameNode don't track a separate notion of creation time, just mtime and atime. Is there something I've missed? what operations on child entries update the modtime? mkdir, create, delete, rename. And which don't? Yes, and to this please also add concat. chmod and set time calls? chmod (and setacl) and setTimes do not alter any modification times, neither on the target path itself nor its parent.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          thanks

          Show
          stevel@apache.org Steve Loughran added a comment - thanks
          Hide
          stevel@apache.org Steve Loughran added a comment -

          patch 003. Make it even more clear we aren't sure what is happening. Add that rename() often/usually/may update the timestamps (ish)

          Show
          stevel@apache.org Steve Loughran added a comment - patch 003. Make it even more clear we aren't sure what is happening. Add that rename() often/usually/may update the timestamps (ish)
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 mvninstall 12m 48s trunk passed
          +1 mvnsite 1m 0s trunk passed
          +1 mvnsite 0m 54s the patch passed
          -1 whitespace 0m 0s The patch has 3 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          +1 asflicense 0m 19s The patch does not generate ASF License warnings.
          15m 36s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-13946
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12848909/HADOOP-13946-003.patch
          Optional Tests asflicense mvnsite
          uname Linux 2d91012570ca 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 3fa0d54
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/11495/artifact/patchprocess/whitespace-eol.txt
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11495/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 mvninstall 12m 48s trunk passed +1 mvnsite 1m 0s trunk passed +1 mvnsite 0m 54s the patch passed -1 whitespace 0m 0s The patch has 3 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 asflicense 0m 19s The patch does not generate ASF License warnings. 15m 36s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13946 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12848909/HADOOP-13946-003.patch Optional Tests asflicense mvnsite uname Linux 2d91012570ca 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 3fa0d54 whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/11495/artifact/patchprocess/whitespace-eol.txt modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11495/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          patch 004

          • be more specific about HDFS, including its atime granularity
          • mention where other filesystems are likely to vary
          • be even vaguer about object stores

          I think I've addressed chris's comments, and I've reached the limits of my understanding, even with a bit more rummaging round the source tree.

          Show
          stevel@apache.org Steve Loughran added a comment - patch 004 be more specific about HDFS, including its atime granularity mention where other filesystems are likely to vary be even vaguer about object stores I think I've addressed chris's comments, and I've reached the limits of my understanding, even with a bit more rummaging round the source tree.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 12s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 mvninstall 12m 30s trunk passed
          +1 mvnsite 0m 55s trunk passed
          +1 mvnsite 0m 50s the patch passed
          -1 whitespace 0m 0s The patch has 6 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          +1 asflicense 0m 17s The patch does not generate ASF License warnings.
          15m 8s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-13946
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12848924/HADOOP-13946-004.patch
          Optional Tests asflicense mvnsite
          uname Linux bc5550b0b49e 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 3fa0d54
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/11496/artifact/patchprocess/whitespace-eol.txt
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11496/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 12s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 mvninstall 12m 30s trunk passed +1 mvnsite 0m 55s trunk passed +1 mvnsite 0m 50s the patch passed -1 whitespace 0m 0s The patch has 6 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 asflicense 0m 17s The patch does not generate ASF License warnings. 15m 8s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13946 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12848924/HADOOP-13946-004.patch Optional Tests asflicense mvnsite uname Linux bc5550b0b49e 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 3fa0d54 whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/11496/artifact/patchprocess/whitespace-eol.txt modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11496/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Anyone up for reviewing this?

          Show
          stevel@apache.org Steve Loughran added a comment - Anyone up for reviewing this?
          Hide
          liuml07 Mingliang Liu added a comment -

          I'm +1 on the v4 patch.

          I'll hold on commit for 5 days in case Thomas Demoor or Chris Nauroth has a chance to review.

          Show
          liuml07 Mingliang Liu added a comment - I'm +1 on the v4 patch. I'll hold on commit for 5 days in case Thomas Demoor or Chris Nauroth has a chance to review.
          Hide
          cnauroth Chris Nauroth added a comment -

          The content looks great. I have a few minor copy editing comments.

          * When a file is renamed, it's modification time is not changed, *but the source
            and destination directories have their modification times updated.
          

          Replace with "its modification time". I think you were going for italics with that '*', but it's unclosed and rendering the actual '*' into the page when I build the site.

            the default granularity is 1 houre. If the precision set to zero, access times
          

          Change to "hours" and "precision is set to zero".

            `create()` call, or the actual the time which the PUT request was initiated.
          

          Remove the extra "the".

           * `FileSystem.chmod()` may update modification times (example: Azure wasb://).
          

          Markdown is rendering an actualy hyperlink that targets "wasb://)", which does nothing when clicked. Can this be suppressed or changed to a link to our WASB document?

          After that, this patch will be ready to go.

          Show
          cnauroth Chris Nauroth added a comment - The content looks great. I have a few minor copy editing comments. * When a file is renamed, it's modification time is not changed, *but the source and destination directories have their modification times updated. Replace with "its modification time". I think you were going for italics with that '*', but it's unclosed and rendering the actual '*' into the page when I build the site. the default granularity is 1 houre. If the precision set to zero, access times Change to "hours" and "precision is set to zero". `create()` call, or the actual the time which the PUT request was initiated. Remove the extra "the". * `FileSystem.chmod()` may update modification times (example: Azure wasb: //). Markdown is rendering an actualy hyperlink that targets "wasb://)", which does nothing when clicked. Can this be suppressed or changed to a link to our WASB document? After that, this patch will be ready to go.
          Hide
          Thomas Demoor Thomas Demoor added a comment -

          my previous remarks have been addressed. didn't find anything more in my run through.
          +1

          Show
          Thomas Demoor Thomas Demoor added a comment - my previous remarks have been addressed. didn't find anything more in my run through. +1
          Hide
          liuml07 Mingliang Liu added a comment - - edited

          Chris' comment makes sense to me. I slightly changed the v4 patch and uploaded the v5 one for addressing that. Specially, the *but was changed to "but" removing the "*". I think we can live without that. For wasb:// linking problem I used ` to wrap it as code as `wasb://`. Also fixed typo "artificially".

          Preview from my screenshot Screen Shot 2017-03-07 at 3.44.43 PM.png.

          I'm assuming the v5 patch inherits all the +1s on v4 patch; will commit in 3 days if no objections.

          Show
          liuml07 Mingliang Liu added a comment - - edited Chris' comment makes sense to me. I slightly changed the v4 patch and uploaded the v5 one for addressing that. Specially, the *but was changed to "but" removing the "*". I think we can live without that. For wasb:// linking problem I used ` to wrap it as code as `wasb://`. Also fixed typo "artificially". Preview from my screenshot Screen Shot 2017-03-07 at 3.44.43 PM.png . I'm assuming the v5 patch inherits all the +1s on v4 patch; will commit in 3 days if no objections.
          Hide
          liuml07 Mingliang Liu added a comment -

          Committed to trunk, branch-2 and branch-2.8 branches. Thanks Thomas Demoor and Chris Nauroth for review and helpful comments. Thanks Steve Loughran for the contribution.

          Show
          liuml07 Mingliang Liu added a comment - Committed to trunk , branch-2 and branch-2.8 branches. Thanks Thomas Demoor and Chris Nauroth for review and helpful comments. Thanks Steve Loughran for the contribution.

            People

            • Assignee:
              stevel@apache.org Steve Loughran
              Reporter:
              stevel@apache.org Steve Loughran
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development