Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-3485

DataTransferThrottler will over-throttle when currentTimeMillis jumps

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 2.0.2-alpha
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      When the system clock is set backwards, DataTransferThrottler will simply pause until the clock reaches the end of the previously calculated transfer period:

          this.curPeriodStart = System.currentTimeMillis();
      ...
          while (curReserve <= 0) {
            long now = System.currentTimeMillis();
            long curPeriodEnd = curPeriodStart + period;
            if ( now < curPeriodEnd ) {
              try {
                wait( curPeriodEnd - now );
      

      Instead of using currentTimeMillis() which is affected by system-clock-changes, this code should use nanoTime which ticks forward monotonically.

      1. hdfs-3485-2.patch
        6 kB
        Andy Isaacson
      2. hdfs-3485.patch
        2 kB
        Andy Isaacson
      3. hdfs-3485.1.patch
        4 kB
        Andy Isaacson

        Activity

        Arun C Murthy made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Arun C Murthy made changes -
        Fix Version/s 2.0.2-alpha [ 12322472 ]
        Fix Version/s 2.1.0-alpha [ 12321440 ]
        Todd Lipcon made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags Reviewed [ 10343 ]
        Fix Version/s 2.0.1-alpha [ 12321440 ]
        Resolution Fixed [ 1 ]
        Andy Isaacson made changes -
        Attachment hdfs-3485-2.patch [ 12530616 ]
        Andy Isaacson made changes -
        Attachment hdfs-3485.1.patch [ 12530582 ]
        Andy Isaacson made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Andy Isaacson made changes -
        Attachment hdfs-3485.patch [ 12530483 ]
        Andy Isaacson made changes -
        Attachment hdfs-3485.patch [ 12530484 ]
        Andy Isaacson made changes -
        Field Original Value New Value
        Attachment hdfs-3485.patch [ 12530483 ]
        Andy Isaacson created issue -

          People

          • Assignee:
            Andy Isaacson
            Reporter:
            Andy Isaacson
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development