Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-13994

Improve DataNode BlockSender waitForMinLength

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 3.2.0
    • Fix Version/s: 3.3.0
    • Component/s: datanode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Flags:
      Patch

      Description

      BlockSender.java
        private static void waitForMinLength(ReplicaInPipeline rbw, long len)
            throws IOException {
          // Wait for 3 seconds for rbw replica to reach the minimum length
          for (int i = 0; i < 30 && rbw.getBytesOnDisk() < len; i++) {
            try {
              Thread.sleep(100);
            } catch (InterruptedException ie) {
              throw new IOException(ie);
            }
          }
          long bytesOnDisk = rbw.getBytesOnDisk();
          if (bytesOnDisk < len) {
            throw new IOException(
                String.format("Need %d bytes, but only %d bytes available", len,
                    bytesOnDisk));
          }
        }
       

      It is not very efficient to poll for status in this way. Instead, use notifyAll within the ReplicaInPipeline to notify the caller when the replica has reached a certain size.

        Attachments

        1. HDFS-13994.5.patch
          9 kB
          David Mollitor
        2. HDFS-13994.4.patch
          9 kB
          David Mollitor
        3. HDFS-13994.3.patch
          9 kB
          David Mollitor
        4. HDFS-13994.2.patch
          9 kB
          David Mollitor
        5. HDFS-13994.1.patch
          9 kB
          David Mollitor

          Activity

            People

            • Assignee:
              belugabehr David Mollitor
              Reporter:
              belugabehr David Mollitor
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: