Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-2041

Some mtimes and atimes are lost when edit logs are replayed

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.23.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The refactoring in HDFS-2003 allowed findbugs to expose two potential bugs:

      • the atime field logged with OP_MKDIR is unused
      • the timestamp field logged with OP_CONCAT_DELETE is unused

      The concat issue is definitely real. The atime for MKDIR might always be identical to mtime in that case, in which case it could be ignored.

      1. hdfs-2041.txt
        6 kB
        Todd Lipcon
      2. hdfs-2041.txt
        7 kB
        Todd Lipcon

        Issue Links

          Activity

          Hide
          Todd Lipcon added a comment -

          Patch on top of HDFS-2003 to fix this issue.

          Show
          Todd Lipcon added a comment - Patch on top of HDFS-2003 to fix this issue.
          Hide
          Todd Lipcon added a comment -

          Rebased on trunk

          Show
          Todd Lipcon added a comment - Rebased on trunk
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12481995/hdfs-2041.txt
          against trunk revision 1134124.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these core unit tests:
          org.apache.hadoop.cli.TestHDFSCLI

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/758//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/758//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/758//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12481995/hdfs-2041.txt against trunk revision 1134124. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.cli.TestHDFSCLI +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/758//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/758//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/758//console This message is automatically generated.
          Hide
          Aaron T. Myers added a comment -

          +1, the change looks good to me, Todd. Great find.

          Thanks also for removing the vestigial runCommand from TestHDFSConcat.

          Show
          Aaron T. Myers added a comment - +1, the change looks good to me, Todd. Great find. Thanks also for removing the vestigial runCommand from TestHDFSConcat .
          Hide
          Todd Lipcon added a comment -

          Committed to trunk.

          Even though this bug affects earlier versions, it's not a major issue and the concat feature is rarely used. Since the backport would be hairy, I've elected not to backport for now.

          Thanks for reviewing, Aaron.

          Show
          Todd Lipcon added a comment - Committed to trunk. Even though this bug affects earlier versions, it's not a major issue and the concat feature is rarely used. Since the backport would be hairy, I've elected not to backport for now. Thanks for reviewing, Aaron.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #746 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/746/)

          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #746 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/746/ )
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #699 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/699/)

          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #699 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/699/ )

            People

            • Assignee:
              Todd Lipcon
              Reporter:
              Todd Lipcon
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development