Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-2041

Some mtimes and atimes are lost when edit logs are replayed

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.23.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The refactoring in HDFS-2003 allowed findbugs to expose two potential bugs:

      • the atime field logged with OP_MKDIR is unused
      • the timestamp field logged with OP_CONCAT_DELETE is unused

      The concat issue is definitely real. The atime for MKDIR might always be identical to mtime in that case, in which case it could be ignored.

      1. hdfs-2041.txt
        6 kB
        Todd Lipcon
      2. hdfs-2041.txt
        7 kB
        Todd Lipcon

        Issue Links

          Activity

          Todd Lipcon created issue -
          Hide
          Todd Lipcon added a comment -

          Patch on top of HDFS-2003 to fix this issue.

          Show
          Todd Lipcon added a comment - Patch on top of HDFS-2003 to fix this issue.
          Todd Lipcon made changes -
          Field Original Value New Value
          Attachment hdfs-2041.txt [ 12481641 ]
          Todd Lipcon made changes -
          Link This issue relates to HDFS-2003 [ HDFS-2003 ]
          Aaron T. Myers made changes -
          Assignee Todd Lipcon [ tlipcon ]
          Hide
          Todd Lipcon added a comment -

          Rebased on trunk

          Show
          Todd Lipcon added a comment - Rebased on trunk
          Todd Lipcon made changes -
          Attachment hdfs-2041.txt [ 12481995 ]
          Todd Lipcon made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12481995/hdfs-2041.txt
          against trunk revision 1134124.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these core unit tests:
          org.apache.hadoop.cli.TestHDFSCLI

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/758//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/758//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/758//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12481995/hdfs-2041.txt against trunk revision 1134124. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.cli.TestHDFSCLI +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/758//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/758//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/758//console This message is automatically generated.
          Hide
          Aaron T. Myers added a comment -

          +1, the change looks good to me, Todd. Great find.

          Thanks also for removing the vestigial runCommand from TestHDFSConcat.

          Show
          Aaron T. Myers added a comment - +1, the change looks good to me, Todd. Great find. Thanks also for removing the vestigial runCommand from TestHDFSConcat .
          Hide
          Todd Lipcon added a comment -

          Committed to trunk.

          Even though this bug affects earlier versions, it's not a major issue and the concat feature is rarely used. Since the backport would be hairy, I've elected not to backport for now.

          Thanks for reviewing, Aaron.

          Show
          Todd Lipcon added a comment - Committed to trunk. Even though this bug affects earlier versions, it's not a major issue and the concat feature is rarely used. Since the backport would be hairy, I've elected not to backport for now. Thanks for reviewing, Aaron.
          Todd Lipcon made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Resolution Fixed [ 1 ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #746 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/746/)

          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #746 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/746/ )
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #699 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/699/)

          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #699 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/699/ )
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Patch Available Patch Available
          2d 23h 43m 1 Todd Lipcon 09/Jun/11 23:59
          Patch Available Patch Available Resolved Resolved
          18h 57m 1 Todd Lipcon 10/Jun/11 18:57
          Resolved Resolved Closed Closed
          157d 6h 55m 1 Arun C Murthy 15/Nov/11 00:53

            People

            • Assignee:
              Todd Lipcon
              Reporter:
              Todd Lipcon
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development