Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.0
    • Fix Version/s: 1.2.0
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Let's port HDFS-1457 (configuration option to enable limiting the transfer rate used when sending the image and edits for checkpointing) to branch-1.

      1. hdfs-3515.txt
        19 kB
        Eli Collins

        Issue Links

          Activity

          Eli Collins created issue -
          Hide
          Eli Collins added a comment -

          Patch attached. Straightforward change. I resisted cleaning up BlockTransferThrottler so it's clear it's just a rename to DataTransferThrottler.

          Passes all the unit tests. I'm also currently testing setting dfs.image.transfer.bandwidthPerSec to 1mb/sec with a large image to confirm that the rate is in fact throttled and also to test the corner case where the checkpoint period is shorter than the time it takes to transmit the image.

          Show
          Eli Collins added a comment - Patch attached. Straightforward change. I resisted cleaning up BlockTransferThrottler so it's clear it's just a rename to DataTransferThrottler. Passes all the unit tests. I'm also currently testing setting dfs.image.transfer.bandwidthPerSec to 1mb/sec with a large image to confirm that the rate is in fact throttled and also to test the corner case where the checkpoint period is shorter than the time it takes to transmit the image.
          Eli Collins made changes -
          Field Original Value New Value
          Attachment hdfs-3515.txt [ 12531273 ]
          Hide
          Eli Collins added a comment -

          Test patch results follow. The findbugs warnings are HADOOP-7847.

               [exec] 
               [exec] -1 overall.  
               [exec] 
               [exec]     +1 @author.  The patch does not contain any @author tags.
               [exec] 
               [exec]     +1 tests included.  The patch appears to include 3 new or modified tests.
               [exec] 
               [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
               [exec] 
               [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
               [exec] 
               [exec]     -1 findbugs.  The patch appears to introduce 8 new Findbugs (version 1.3.9) warnings.
               [exec] 
          
          Show
          Eli Collins added a comment - Test patch results follow. The findbugs warnings are HADOOP-7847 . [exec] [exec] -1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 3 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] -1 findbugs. The patch appears to introduce 8 new Findbugs (version 1.3.9) warnings. [exec]
          Hide
          Todd Lipcon added a comment -

          Patch looks good to me. +1 pending the results of your test you described above.

          Show
          Todd Lipcon added a comment - Patch looks good to me. +1 pending the results of your test you described above.
          Eli Collins made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          Hide
          Eli Collins added a comment -

          Thanks Todd. I've confirmed the testing and committed to branch-1.

          Show
          Eli Collins added a comment - Thanks Todd. I've confirmed the testing and committed to branch-1.
          Eli Collins made changes -
          Status In Progress [ 3 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Target Version/s 1.1.1 [ 12321656 ]
          Fix Version/s 1.2.0 [ 12321657 ]
          Resolution Fixed [ 1 ]
          Eli Collins made changes -
          Assignee Eli Collins [ eli2 ] Eli Collins [ eli ]
          Xiaobo Peng made changes -
          Link This issue is blocked by HDFS-4535 [ HDFS-4535 ]
          Xiaobo Peng made changes -
          Link This issue is blocked by HDFS-4535 [ HDFS-4535 ]
          Xiaobo Peng made changes -
          Link This issue relates to HDFS-4535 [ HDFS-4535 ]
          Hide
          Matt Foley added a comment -

          Closed upon release of Hadoop 1.2.0.

          Show
          Matt Foley added a comment - Closed upon release of Hadoop 1.2.0.
          Matt Foley made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Eli Collins
              Reporter:
              Eli Collins
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development