Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-3020

Auto-logSync based on edit log buffer size broken

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 0.22.0, 0.23.0
    • Fix Version/s: 2.0.0-alpha, 0.23.10
    • Component/s: namenode
    • Labels:
      None

      Description

      HDFS-1112 added a feature whereby the edit log automatically calls logSync() if the buffered data crosses a threshold. However, the code checks bufReady.size() rather than bufCurrent.size() – which is incorrect since the writes themselves go into bufCurrent.

      1. hdfs-3020.txt
        3 kB
        Todd Lipcon
      2. hdfs-3020.txt
        3 kB
        Todd Lipcon
      3. hdfs-3020.txt
        3 kB
        Todd Lipcon

        Issue Links

          Activity

          Hide
          Todd Lipcon added a comment -

          Attached patch fixes the issue. I will test this against a real workload that exhibited the issue as well.

          Show
          Todd Lipcon added a comment - Attached patch fixes the issue. I will test this against a real workload that exhibited the issue as well.
          Hide
          Eli Collins added a comment -

          +1 nice test

          Show
          Eli Collins added a comment - +1 nice test
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12516269/hdfs-3020.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          -1 javac. The patch appears to cause tar ant target to fail.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed the unit tests build

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1918//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1918//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12516269/hdfs-3020.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The patch appears to cause tar ant target to fail. +1 eclipse:eclipse. The patch built with eclipse:eclipse. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed the unit tests build +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1918//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1918//console This message is automatically generated.
          Hide
          Suresh Srinivas added a comment -

          Does this mean that the editlog is being synced only during rolling of editlogs?

          Since you are touching this part of the code, can you please remove throwing unnecessary IOException from FSEditlog#getEditLogManifest, FSEditlog#releaseBackupStream.

          This patch should be committed to 0.23.2.

          Show
          Suresh Srinivas added a comment - Does this mean that the editlog is being synced only during rolling of editlogs? Since you are touching this part of the code, can you please remove throwing unnecessary IOException from FSEditlog#getEditLogManifest, FSEditlog#releaseBackupStream. This patch should be committed to 0.23.2.
          Hide
          Todd Lipcon added a comment -

          Does this mean that the editlog is being synced only during rolling of editlogs?

          Nope, it's being synced OK for any operations that call logSync. It's just that the functionality of HDFS-1112 isn't working – so, for example, if you start a NN and then only perform getBlockLocations calls, it may fill up the log buffer until the NN is entirely out of RAM.

          I'll update the patch against trunk - the previous patch was against HA branch but I forgot that the APIs have been changed a little bit, so the test didn't compile.

          Show
          Todd Lipcon added a comment - Does this mean that the editlog is being synced only during rolling of editlogs? Nope, it's being synced OK for any operations that call logSync . It's just that the functionality of HDFS-1112 isn't working – so, for example, if you start a NN and then only perform getBlockLocations calls, it may fill up the log buffer until the NN is entirely out of RAM. I'll update the patch against trunk - the previous patch was against HA branch but I forgot that the APIs have been changed a little bit, so the test didn't compile.
          Hide
          Todd Lipcon added a comment -

          Updated patch against trunk (changed FSEditLog.openForWrite() to FSEditLog.open())

          Show
          Todd Lipcon added a comment - Updated patch against trunk (changed FSEditLog.openForWrite() to FSEditLog.open())
          Hide
          Eli Collins added a comment -

          +1 update looks good

          Show
          Eli Collins added a comment - +1 update looks good
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12516362/hdfs-3020.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in .

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1920//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1920//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12516362/hdfs-3020.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1920//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1920//console This message is automatically generated.
          Hide
          Todd Lipcon added a comment -

          Suresh, mind if I address those unnecessary IOEs separately? I want to keep this patch tightly targeted to avoid unnecessary merge conflicts, etc (applying across 23, trunk, HA)

          Show
          Todd Lipcon added a comment - Suresh, mind if I address those unnecessary IOEs separately? I want to keep this patch tightly targeted to avoid unnecessary merge conflicts, etc (applying across 23, trunk, HA)
          Hide
          Suresh Srinivas added a comment -

          Sounds good

          Show
          Suresh Srinivas added a comment - Sounds good
          Hide
          Todd Lipcon added a comment -

          The spying I did on trunk didn't work for branch-23. Updated patch spys on metrics instead of the stream itself, which applies to both 23 and trunk. The code's otherwise the same.

          Show
          Todd Lipcon added a comment - The spying I did on trunk didn't work for branch-23. Updated patch spys on metrics instead of the stream itself, which applies to both 23 and trunk. The code's otherwise the same.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12516591/hdfs-3020.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1930//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12516591/hdfs-3020.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1930//console This message is automatically generated.
          Hide
          Eli Collins added a comment -

          +1 updated patch lgtm

          Show
          Eli Collins added a comment - +1 updated patch lgtm
          Hide
          Todd Lipcon added a comment -

          Committed for 23 and trunk. I re-ran the new test on both branches before commit.

          Show
          Todd Lipcon added a comment - Committed for 23 and trunk. I re-ran the new test on both branches before commit.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #1885 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1885/)
          HDFS-3020. Fix editlog to automatically sync when buffer is full. Contributed by Todd Lipcon. (Revision 1295239)

          Result = SUCCESS
          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1295239
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditsDoubleBuffer.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1885 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1885/ ) HDFS-3020 . Fix editlog to automatically sync when buffer is full. Contributed by Todd Lipcon. (Revision 1295239) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1295239 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditsDoubleBuffer.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Commit #608 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/608/)
          HDFS-3020. Fix editlog to automatically sync when buffer is full. Contributed by Todd Lipcon. (Revision 1295241)

          Result = SUCCESS
          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1295241
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditsDoubleBuffer.java
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Commit #608 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/608/ ) HDFS-3020 . Fix editlog to automatically sync when buffer is full. Contributed by Todd Lipcon. (Revision 1295241) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1295241 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditsDoubleBuffer.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #1811 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1811/)
          HDFS-3020. Fix editlog to automatically sync when buffer is full. Contributed by Todd Lipcon. (Revision 1295239)

          Result = SUCCESS
          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1295239
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditsDoubleBuffer.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1811 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1811/ ) HDFS-3020 . Fix editlog to automatically sync when buffer is full. Contributed by Todd Lipcon. (Revision 1295239) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1295239 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditsDoubleBuffer.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-0.23-Commit #620 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/620/)
          HDFS-3020. Fix editlog to automatically sync when buffer is full. Contributed by Todd Lipcon. (Revision 1295241)

          Result = SUCCESS
          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1295241
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditsDoubleBuffer.java
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-0.23-Commit #620 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/620/ ) HDFS-3020 . Fix editlog to automatically sync when buffer is full. Contributed by Todd Lipcon. (Revision 1295241) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1295241 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditsDoubleBuffer.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-0.23-Commit #620 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/620/)
          HDFS-3020. Fix editlog to automatically sync when buffer is full. Contributed by Todd Lipcon. (Revision 1295241)

          Result = ABORTED
          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1295241
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditsDoubleBuffer.java
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Commit #620 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/620/ ) HDFS-3020 . Fix editlog to automatically sync when buffer is full. Contributed by Todd Lipcon. (Revision 1295241) Result = ABORTED todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1295241 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditsDoubleBuffer.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #1818 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1818/)
          HDFS-3020. Fix editlog to automatically sync when buffer is full. Contributed by Todd Lipcon. (Revision 1295239)

          Result = ABORTED
          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1295239
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditsDoubleBuffer.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1818 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1818/ ) HDFS-3020 . Fix editlog to automatically sync when buffer is full. Contributed by Todd Lipcon. (Revision 1295239) Result = ABORTED todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1295239 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditsDoubleBuffer.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #971 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/971/)
          HDFS-3020. Fix editlog to automatically sync when buffer is full. Contributed by Todd Lipcon. (Revision 1295239)

          Result = SUCCESS
          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1295239
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditsDoubleBuffer.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #971 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/971/ ) HDFS-3020 . Fix editlog to automatically sync when buffer is full. Contributed by Todd Lipcon. (Revision 1295239) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1295239 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditsDoubleBuffer.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1006 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1006/)
          HDFS-3020. Fix editlog to automatically sync when buffer is full. Contributed by Todd Lipcon. (Revision 1295239)

          Result = SUCCESS
          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1295239
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditsDoubleBuffer.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1006 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1006/ ) HDFS-3020 . Fix editlog to automatically sync when buffer is full. Contributed by Todd Lipcon. (Revision 1295239) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1295239 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditsDoubleBuffer.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java
          Hide
          Tsz Wo Nicholas Sze added a comment -
          • Why there are no Jerkins report for the latest patch?
          • It looks like the patch introduced a findbugs warning. If it is the case, please revert it.
          Show
          Tsz Wo Nicholas Sze added a comment - Why there are no Jerkins report for the latest patch? It looks like the patch introduced a findbugs warning . If it is the case, please revert it.
          Hide
          Todd Lipcon added a comment -

          The latest patch was against a non-trunk branch so it didn't apply quite cleanly.
          I fixed this findbugs error already on the HA branch which we will merge soon.

          Show
          Todd Lipcon added a comment - The latest patch was against a non-trunk branch so it didn't apply quite cleanly. I fixed this findbugs error already on the HA branch which we will merge soon.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          > The latest patch was against a non-trunk branch so it didn't apply quite cleanly.

          Then, which one is the committed patch for trunk?

          Show
          Tsz Wo Nicholas Sze added a comment - > The latest patch was against a non-trunk branch so it didn't apply quite cleanly. Then, which one is the committed patch for trunk?
          Hide
          Kihwal Lee added a comment -

          Pulled into branch-0.23.

          Show
          Kihwal Lee added a comment - Pulled into branch-0.23.
          Hide
          Hudson added a comment -

          ABORTED: Integrated in Hadoop-Hdfs-0.23-Build #694 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/694/)
          HDFS-3020. Fix editlog to automatically sync when buffer is full. Contributed by Todd Lipcon. (kihwal: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1511968)

          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditsDoubleBuffer.java
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java
          Show
          Hudson added a comment - ABORTED: Integrated in Hadoop-Hdfs-0.23-Build #694 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/694/ ) HDFS-3020 . Fix editlog to automatically sync when buffer is full. Contributed by Todd Lipcon. (kihwal: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1511968 ) /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditsDoubleBuffer.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-0.23-Build #695 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/695/)
          HDFS-3020. Fix editlog to automatically sync when buffer is full. Contributed by Todd Lipcon. (kihwal: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1511968)

          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditsDoubleBuffer.java
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-0.23-Build #695 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/695/ ) HDFS-3020 . Fix editlog to automatically sync when buffer is full. Contributed by Todd Lipcon. (kihwal: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1511968 ) /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditsDoubleBuffer.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java

            People

            • Assignee:
              Todd Lipcon
              Reporter:
              Todd Lipcon
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development