Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-3485

DataTransferThrottler will over-throttle when currentTimeMillis jumps

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 2.0.2-alpha
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      When the system clock is set backwards, DataTransferThrottler will simply pause until the clock reaches the end of the previously calculated transfer period:

          this.curPeriodStart = System.currentTimeMillis();
      ...
          while (curReserve <= 0) {
            long now = System.currentTimeMillis();
            long curPeriodEnd = curPeriodStart + period;
            if ( now < curPeriodEnd ) {
              try {
                wait( curPeriodEnd - now );
      

      Instead of using currentTimeMillis() which is affected by system-clock-changes, this code should use nanoTime which ticks forward monotonically.

        Attachments

        1. hdfs-3485.1.patch
          4 kB
          Andy Isaacson
        2. hdfs-3485.patch
          2 kB
          Andy Isaacson
        3. hdfs-3485-2.patch
          6 kB
          Andy Isaacson

          Activity

            People

            • Assignee:
              adi2 Andy Isaacson
              Reporter:
              adi2 Andy Isaacson
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: