Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.0
    • Fix Version/s: 1.2.0
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Let's port HDFS-1457 (configuration option to enable limiting the transfer rate used when sending the image and edits for checkpointing) to branch-1.

      1. hdfs-3515.txt
        19 kB
        Eli Collins

        Issue Links

          Activity

          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open In Progress In Progress
          1d 7h 30m 1 Eli Collins 09/Jun/12 01:37
          In Progress In Progress Resolved Resolved
          20s 1 Eli Collins 09/Jun/12 01:37
          Resolved Resolved Closed Closed
          340d 4h 38m 1 Matt Foley 15/May/13 06:15
          Matt Foley made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Hide
          Matt Foley added a comment -

          Closed upon release of Hadoop 1.2.0.

          Show
          Matt Foley added a comment - Closed upon release of Hadoop 1.2.0.
          Xiaobo Peng made changes -
          Link This issue relates to HDFS-4535 [ HDFS-4535 ]
          Xiaobo Peng made changes -
          Link This issue is blocked by HDFS-4535 [ HDFS-4535 ]
          Xiaobo Peng made changes -
          Link This issue is blocked by HDFS-4535 [ HDFS-4535 ]
          Eli Collins made changes -
          Assignee Eli Collins [ eli2 ] Eli Collins [ eli ]
          Eli Collins made changes -
          Status In Progress [ 3 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Target Version/s 1.1.1 [ 12321656 ]
          Fix Version/s 1.2.0 [ 12321657 ]
          Resolution Fixed [ 1 ]
          Hide
          Eli Collins added a comment -

          Thanks Todd. I've confirmed the testing and committed to branch-1.

          Show
          Eli Collins added a comment - Thanks Todd. I've confirmed the testing and committed to branch-1.
          Eli Collins made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          Hide
          Todd Lipcon added a comment -

          Patch looks good to me. +1 pending the results of your test you described above.

          Show
          Todd Lipcon added a comment - Patch looks good to me. +1 pending the results of your test you described above.
          Hide
          Eli Collins added a comment -

          Test patch results follow. The findbugs warnings are HADOOP-7847.

               [exec] 
               [exec] -1 overall.  
               [exec] 
               [exec]     +1 @author.  The patch does not contain any @author tags.
               [exec] 
               [exec]     +1 tests included.  The patch appears to include 3 new or modified tests.
               [exec] 
               [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
               [exec] 
               [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
               [exec] 
               [exec]     -1 findbugs.  The patch appears to introduce 8 new Findbugs (version 1.3.9) warnings.
               [exec] 
          
          Show
          Eli Collins added a comment - Test patch results follow. The findbugs warnings are HADOOP-7847 . [exec] [exec] -1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 3 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] -1 findbugs. The patch appears to introduce 8 new Findbugs (version 1.3.9) warnings. [exec]
          Eli Collins made changes -
          Field Original Value New Value
          Attachment hdfs-3515.txt [ 12531273 ]
          Hide
          Eli Collins added a comment -

          Patch attached. Straightforward change. I resisted cleaning up BlockTransferThrottler so it's clear it's just a rename to DataTransferThrottler.

          Passes all the unit tests. I'm also currently testing setting dfs.image.transfer.bandwidthPerSec to 1mb/sec with a large image to confirm that the rate is in fact throttled and also to test the corner case where the checkpoint period is shorter than the time it takes to transmit the image.

          Show
          Eli Collins added a comment - Patch attached. Straightforward change. I resisted cleaning up BlockTransferThrottler so it's clear it's just a rename to DataTransferThrottler. Passes all the unit tests. I'm also currently testing setting dfs.image.transfer.bandwidthPerSec to 1mb/sec with a large image to confirm that the rate is in fact throttled and also to test the corner case where the checkpoint period is shorter than the time it takes to transmit the image.
          Eli Collins created issue -

            People

            • Assignee:
              Eli Collins
              Reporter:
              Eli Collins
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development