Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-1572

Checkpointer should trigger checkpoint with specified period.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.22.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

        long now = now();
        boolean shouldCheckpoint = false;
        if(now >= lastCheckpointTime + periodMSec) {
          shouldCheckpoint = true;
        } else {
          long size = getJournalSize();
          if(size >= checkpointSize)
            shouldCheckpoint = true;
        }
      
      {dfs.namenode.checkpoint.period}

      in configuration determines the period of checkpoint. However, with above code, the Checkpointer triggers a checkpoint every 5 minutes (periodMSec=5*60*1000). According to SecondaryNameNode.java, the first if statement should be:

      if(now >= lastCheckpointTime + 1000 * checkpointPeriod) {
       
      1. 1527-1.diff
        0.6 kB
        Liyin Liang
      2. 1572-2.diff
        3 kB
        Liyin Liang
      3. HDFS-1572.patch
        2 kB
        Jakob Homan
      4. HDFS-1572-2.patch
        2 kB
        Jakob Homan
      5. HDFS-1572-3.patch
        2 kB
        Jakob Homan

        Issue Links

          Activity

          Liyin Liang created issue -
          Hide
          Liyin Liang added a comment -

          patch for trunk.

          Show
          Liyin Liang added a comment - patch for trunk.
          Liyin Liang made changes -
          Field Original Value New Value
          Attachment 1527-1.diff [ 12467633 ]
          Hide
          Jakob Homan added a comment -

          The code is definitely wrong, but I think the entire method can be refactored to be more readable and clear. Right now it's a bit of a mess. I've attached a patch which I think does this. Liyin, what do you think?

          Show
          Jakob Homan added a comment - The code is definitely wrong, but I think the entire method can be refactored to be more readable and clear. Right now it's a bit of a mess. I've attached a patch which I think does this. Liyin, what do you think?
          Jakob Homan made changes -
          Attachment HDFS-1572.patch [ 12467781 ]
          Hide
          Jakob Homan added a comment -

          Submitting patch to Hudson. Unfortunately, this code is very difficult to reach in a unit test. We should work on refactoring it to make it more testable. I think the code is now straight forward enough to verify correctness by hand and via existing tests.

          Show
          Jakob Homan added a comment - Submitting patch to Hudson. Unfortunately, this code is very difficult to reach in a unit test. We should work on refactoring it to make it more testable. I think the code is now straight forward enough to verify correctness by hand and via existing tests.
          Jakob Homan made changes -
          Fix Version/s 0.21.0 [ 12314046 ]
          Status Open [ 1 ] Patch Available [ 10002 ]
          Affects Version/s 0.22.0 [ 12314241 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12467781/HDFS-1572.patch
          against trunk revision 1056206.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these core unit tests:
          org.apache.hadoop.fs.permission.TestStickyBit
          org.apache.hadoop.hdfs.security.TestDelegationToken
          org.apache.hadoop.hdfs.server.common.TestDistributedUpgrade
          org.apache.hadoop.hdfs.server.datanode.TestBlockReport
          org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
          org.apache.hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics
          org.apache.hadoop.hdfs.server.namenode.TestBackupNode
          org.apache.hadoop.hdfs.server.namenode.TestBlocksWithNotEnoughRacks
          org.apache.hadoop.hdfs.server.namenode.TestBlockTokenWithDFS
          org.apache.hadoop.hdfs.server.namenode.TestCheckpoint
          org.apache.hadoop.hdfs.server.namenode.TestFsck
          org.apache.hadoop.hdfs.server.namenode.TestNameEditsConfigs
          org.apache.hadoop.hdfs.server.namenode.TestStorageRestore
          org.apache.hadoop.hdfs.TestCrcCorruption
          org.apache.hadoop.hdfs.TestDatanodeBlockScanner
          org.apache.hadoop.hdfs.TestDatanodeDeath
          org.apache.hadoop.hdfs.TestDFSClientRetries
          org.apache.hadoop.hdfs.TestDFSFinalize
          org.apache.hadoop.hdfs.TestDFSRollback
          org.apache.hadoop.hdfs.TestDFSShell
          org.apache.hadoop.hdfs.TestDFSStartupVersions
          org.apache.hadoop.hdfs.TestDFSStorageStateRecovery
          org.apache.hadoop.hdfs.TestDFSUpgradeFromImage
          org.apache.hadoop.hdfs.TestDFSUpgrade
          org.apache.hadoop.hdfs.TestDistributedFileSystem
          org.apache.hadoop.hdfs.TestFileAppend2
          org.apache.hadoop.hdfs.TestFileAppend3
          org.apache.hadoop.hdfs.TestFileAppend4
          org.apache.hadoop.hdfs.TestFileAppend
          org.apache.hadoop.hdfs.TestFileConcurrentReader
          org.apache.hadoop.hdfs.TestFileCreationNamenodeRestart
          org.apache.hadoop.hdfs.TestFileCreation
          org.apache.hadoop.hdfs.TestHDFSFileSystemContract
          org.apache.hadoop.hdfs.TestHDFSTrash
          org.apache.hadoop.hdfs.TestPread
          org.apache.hadoop.hdfs.TestQuota
          org.apache.hadoop.hdfs.TestReplication
          org.apache.hadoop.hdfs.TestRestartDFS
          org.apache.hadoop.hdfs.TestSetrepDecreasing
          org.apache.hadoop.hdfs.TestSetrepIncreasing
          org.apache.hadoop.hdfs.TestWriteConfigurationToDFS

          -1 contrib tests. The patch failed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/92//testReport/
          Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/92//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/92//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12467781/HDFS-1572.patch against trunk revision 1056206. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.fs.permission.TestStickyBit org.apache.hadoop.hdfs.security.TestDelegationToken org.apache.hadoop.hdfs.server.common.TestDistributedUpgrade org.apache.hadoop.hdfs.server.datanode.TestBlockReport org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting org.apache.hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics org.apache.hadoop.hdfs.server.namenode.TestBackupNode org.apache.hadoop.hdfs.server.namenode.TestBlocksWithNotEnoughRacks org.apache.hadoop.hdfs.server.namenode.TestBlockTokenWithDFS org.apache.hadoop.hdfs.server.namenode.TestCheckpoint org.apache.hadoop.hdfs.server.namenode.TestFsck org.apache.hadoop.hdfs.server.namenode.TestNameEditsConfigs org.apache.hadoop.hdfs.server.namenode.TestStorageRestore org.apache.hadoop.hdfs.TestCrcCorruption org.apache.hadoop.hdfs.TestDatanodeBlockScanner org.apache.hadoop.hdfs.TestDatanodeDeath org.apache.hadoop.hdfs.TestDFSClientRetries org.apache.hadoop.hdfs.TestDFSFinalize org.apache.hadoop.hdfs.TestDFSRollback org.apache.hadoop.hdfs.TestDFSShell org.apache.hadoop.hdfs.TestDFSStartupVersions org.apache.hadoop.hdfs.TestDFSStorageStateRecovery org.apache.hadoop.hdfs.TestDFSUpgradeFromImage org.apache.hadoop.hdfs.TestDFSUpgrade org.apache.hadoop.hdfs.TestDistributedFileSystem org.apache.hadoop.hdfs.TestFileAppend2 org.apache.hadoop.hdfs.TestFileAppend3 org.apache.hadoop.hdfs.TestFileAppend4 org.apache.hadoop.hdfs.TestFileAppend org.apache.hadoop.hdfs.TestFileConcurrentReader org.apache.hadoop.hdfs.TestFileCreationNamenodeRestart org.apache.hadoop.hdfs.TestFileCreation org.apache.hadoop.hdfs.TestHDFSFileSystemContract org.apache.hadoop.hdfs.TestHDFSTrash org.apache.hadoop.hdfs.TestPread org.apache.hadoop.hdfs.TestQuota org.apache.hadoop.hdfs.TestReplication org.apache.hadoop.hdfs.TestRestartDFS org.apache.hadoop.hdfs.TestSetrepDecreasing org.apache.hadoop.hdfs.TestSetrepIncreasing org.apache.hadoop.hdfs.TestWriteConfigurationToDFS -1 contrib tests. The patch failed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/92//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/92//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/92//console This message is automatically generated.
          Hide
          Liyin Liang added a comment -

          Hi Jakob,
          Thanks for you work and advice. We can move some logic to static functions to make it more testable. I'll attach a patch with a simple test case. Any thoughts?

          Show
          Liyin Liang added a comment - Hi Jakob, Thanks for you work and advice. We can move some logic to static functions to make it more testable. I'll attach a patch with a simple test case. Any thoughts?
          Liyin Liang made changes -
          Attachment 1572-2.diff [ 12467826 ]
          Hide
          Jakob Homan added a comment -

          Liyin-
          I'd rather not duplicate a bunch of the logic in the test. Were we to put out the effort into testing, I'd rather go ahead and take an approach like what Project Voldemort has for time-dependent operations: http://s.apache.org/uU. I recently used it to good effect in unit testing a similar bit of code: http://s.apache.org/bMQ It worked quite well.

          That being said, I think the patch I submitted does a more complete job of cleaning up the code in general and I'd like to go ahead with that one. Adding more tests would be great, but is a bigger issue. The failed unit tests seem to be bogus time outs. They're not related to this code and are not reproducing on my local box. I'm running the full test suite now and will post results when they finish.

          Show
          Jakob Homan added a comment - Liyin- I'd rather not duplicate a bunch of the logic in the test. Were we to put out the effort into testing, I'd rather go ahead and take an approach like what Project Voldemort has for time-dependent operations: http://s.apache.org/uU . I recently used it to good effect in unit testing a similar bit of code: http://s.apache.org/bMQ It worked quite well. That being said, I think the patch I submitted does a more complete job of cleaning up the code in general and I'd like to go ahead with that one. Adding more tests would be great, but is a bigger issue. The failed unit tests seem to be bogus time outs. They're not related to this code and are not reproducing on my local box. I'm running the full test suite now and will post results when they finish.
          Hide
          Jakob Homan added a comment -

          Updated patch so checkpointer only sleeps for a minute between running rather than five. This means the checkpoint time setting will be delayed by a maximum of a minute. Ran tests locally, all pass except known bad.

          Show
          Jakob Homan added a comment - Updated patch so checkpointer only sleeps for a minute between running rather than five. This means the checkpoint time setting will be delayed by a maximum of a minute. Ran tests locally, all pass except known bad.
          Jakob Homan made changes -
          Attachment HDFS-1572-2.patch [ 12467950 ]
          Jakob Homan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Jakob Homan added a comment -

          re-triggering hudson.

          Show
          Jakob Homan added a comment - re-triggering hudson.
          Jakob Homan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Boris Shkolnik added a comment -

          getJournalSize() is an RPC call. I think we should make the call only if periodExpired is not true..
          The way it is done in the patch it will be called every minute (BTW why you changed sleep(period) to sleep (60*1000)?)

          Show
          Boris Shkolnik added a comment - getJournalSize() is an RPC call. I think we should make the call only if periodExpired is not true.. The way it is done in the patch it will be called every minute (BTW why you changed sleep(period) to sleep (60*1000)?)
          Hide
          Jakob Homan added a comment -

          The getJournalSize() rpc is only called once every five minutes to check if the journal size is too big. The 60 seconds is just to check if that call needs to be made, or if the duration-based checkpointing should be done. The 60 seconds improves the granularity of the duration. Currently it's duration+5 minutes at a maximum. This moves it to duration+1 minute at a maximum. The same number of RPC calls are being made since the getJournalSize() branch is only traversed every 5 minutes. It is true that if the period has expired (once an hour by default, once a day on all production systems I've seen), we'd do a single extra RPC call. I think this is worth it to improve code clarity.

          Show
          Jakob Homan added a comment - The getJournalSize() rpc is only called once every five minutes to check if the journal size is too big. The 60 seconds is just to check if that call needs to be made, or if the duration-based checkpointing should be done. The 60 seconds improves the granularity of the duration. Currently it's duration+5 minutes at a maximum. This moves it to duration+1 minute at a maximum. The same number of RPC calls are being made since the getJournalSize() branch is only traversed every 5 minutes. It is true that if the period has expired (once an hour by default, once a day on all production systems I've seen), we'd do a single extra RPC call. I think this is worth it to improve code clarity.
          Hide
          Boris Shkolnik added a comment -

          +1

          Show
          Boris Shkolnik added a comment - +1
          Hide
          Konstantin Shvachko added a comment -

          Jakob, the idea of the original code was to wake up every 5 minutes or checkpointPeriod, whichever is less. Then verify the edits length and checkpoint if necessary. Your patch achieves the same functionality but wakes the thread more often and needs more variables.
          Hard-coded 60 seconds is not good if I want to checkpoint every 30 seconds, which I need for debugging. checkpointPeriod set to 30 sec should overrule 60 secs. Does it make sense?

          Show
          Konstantin Shvachko added a comment - Jakob, the idea of the original code was to wake up every 5 minutes or checkpointPeriod, whichever is less. Then verify the edits length and checkpoint if necessary. Your patch achieves the same functionality but wakes the thread more often and needs more variables. Hard-coded 60 seconds is not good if I want to checkpoint every 30 seconds, which I need for debugging. checkpointPeriod set to 30 sec should overrule 60 secs. Does it make sense?
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12467826/1572-2.diff
          against trunk revision 1056206.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these core unit tests:

          -1 contrib tests. The patch failed contrib unit tests.

          -1 system test framework. The patch failed system test framework compile.

          Test results: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/93//testReport/
          Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/93//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/93//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12467826/1572-2.diff against trunk revision 1056206. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: -1 contrib tests. The patch failed contrib unit tests. -1 system test framework. The patch failed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/93//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/93//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/93//console This message is automatically generated.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12467950/HDFS-1572-2.patch
          against trunk revision 1057414.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these core unit tests:
          org.apache.hadoop.hdfs.server.namenode.TestStorageRestore
          org.apache.hadoop.hdfs.TestFileConcurrentReader

          -1 contrib tests. The patch failed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/100//testReport/
          Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/100//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/100//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12467950/HDFS-1572-2.patch against trunk revision 1057414. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.hdfs.server.namenode.TestStorageRestore org.apache.hadoop.hdfs.TestFileConcurrentReader -1 contrib tests. The patch failed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/100//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/100//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/100//console This message is automatically generated.
          Hide
          Jakob Homan added a comment -

          Good catch Konstantin. Changed the sleep time to be the minimum of time-based or size-based intervals. This way, regardless of whichever is lower, we'll sleep for the minimum time needed.

          Show
          Jakob Homan added a comment - Good catch Konstantin. Changed the sleep time to be the minimum of time-based or size-based intervals. This way, regardless of whichever is lower, we'll sleep for the minimum time needed.
          Jakob Homan made changes -
          Attachment HDFS-1572-3.patch [ 12468291 ]
          Jakob Homan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Jakob Homan added a comment -

          Hudson!

          Show
          Jakob Homan added a comment - Hudson!
          Jakob Homan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Liyin Liang made changes -
          Link This issue relates to HADOOP-4539 [ HADOOP-4539 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12468291/HDFS-1572-3.patch
          against trunk revision 1058402.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these core unit tests:
          org.apache.hadoop.hdfs.server.namenode.TestStorageRestore
          org.apache.hadoop.hdfs.TestFileConcurrentReader

          -1 contrib tests. The patch failed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/109//testReport/
          Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/109//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/109//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12468291/HDFS-1572-3.patch against trunk revision 1058402. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.hdfs.server.namenode.TestStorageRestore org.apache.hadoop.hdfs.TestFileConcurrentReader -1 contrib tests. The patch failed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/109//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/109//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/109//console This message is automatically generated.
          Hide
          Konstantin Shvachko added a comment -

          +1 Looks good to me.

          Show
          Konstantin Shvachko added a comment - +1 Looks good to me.
          Hide
          Jakob Homan added a comment -

          I've committed this to trunk and 22. Resolving.

          Show
          Jakob Homan added a comment - I've committed this to trunk and 22. Resolving.
          Jakob Homan made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Assignee Jakob Homan [ jghoman ]
          Fix Version/s 0.22.0 [ 12314241 ]
          Fix Version/s 0.21.0 [ 12314046 ]
          Resolution Fixed [ 1 ]
          Hide
          Ashwin Giridharan added a comment -

          As per the current solution, if 'periodMS' is configured as '6 mins', then checkpoint will happen only at 10th minute , because of 'fiveMinMS'. Instead we can optimize these two default values, such that the wait will be happening for only LCM('periodMS','fiveMinMS') period , rather than MIN('periodMS','fiveMinMS')

          Show
          Ashwin Giridharan added a comment - As per the current solution, if 'periodMS' is configured as '6 mins', then checkpoint will happen only at 10th minute , because of 'fiveMinMS'. Instead we can optimize these two default values, such that the wait will be happening for only LCM('periodMS','fiveMinMS') period , rather than MIN('periodMS','fiveMinMS')
          Hide
          Ashwin Giridharan added a comment -

          Sorry.. LCM should not be used. We should be using GCD...

          Show
          Ashwin Giridharan added a comment - Sorry.. LCM should not be used. We should be using GCD...
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #643 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/643/)

          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #643 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/643/ )
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-22-branch #35 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-22-branch/35/)

          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-22-branch #35 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-22-branch/35/ )
          Konstantin Shvachko made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Jakob Homan
              Reporter:
              Liyin Liang
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development