Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-3485

DataTransferThrottler will over-throttle when currentTimeMillis jumps

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 2.0.2-alpha
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      When the system clock is set backwards, DataTransferThrottler will simply pause until the clock reaches the end of the previously calculated transfer period:

          this.curPeriodStart = System.currentTimeMillis();
      ...
          while (curReserve <= 0) {
            long now = System.currentTimeMillis();
            long curPeriodEnd = curPeriodStart + period;
            if ( now < curPeriodEnd ) {
              try {
                wait( curPeriodEnd - now );
      

      Instead of using currentTimeMillis() which is affected by system-clock-changes, this code should use nanoTime which ticks forward monotonically.

      1. hdfs-3485-2.patch
        6 kB
        Andy Isaacson
      2. hdfs-3485.1.patch
        4 kB
        Andy Isaacson
      3. hdfs-3485.patch
        2 kB
        Andy Isaacson

        Activity

          People

          • Assignee:
            Andy Isaacson
            Reporter:
            Andy Isaacson
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development