Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-15062

Add LOG when sendIBRs failed

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.3, 3.2.1, 3.1.3
    • Fix Version/s: 3.3.0
    • Component/s: datanode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

        /** Send IBRs to namenode. */
        void sendIBRs(DatanodeProtocol namenode, DatanodeRegistration registration,
            String bpid, String nnRpcLatencySuffix) throws IOException {
          // Generate a list of the pending reports for each storage under the lock
          final StorageReceivedDeletedBlocks[] reports = generateIBRs();
          if (reports.length == 0) {
            // Nothing new to report.
            return;
          }
      
          // Send incremental block reports to the Namenode outside the lock
          if (LOG.isDebugEnabled()) {
            LOG.debug("call blockReceivedAndDeleted: " + Arrays.toString(reports));
          }
          boolean success = false;
          final long startTime = monotonicNow();
          try {
            namenode.blockReceivedAndDeleted(registration, bpid, reports);
            success = true;
          } finally {
      
            if (success) {
              dnMetrics.addIncrementalBlockReport(monotonicNow() - startTime,
                  nnRpcLatencySuffix);
              lastIBR = startTime;
            } else {
              // If we didn't succeed in sending the report, put all of the
              // blocks back onto our queue, but only in the case where we
              // didn't put something newer in the meantime.
              putMissing(reports);
            }
          }
        }
      

      When call namenode.blockReceivedAndDelete failed, will put reports to pendingIBRs. Maybe we should add log for failed case. It is helpful for trouble shooting

        Attachments

        1. HDFS-15062.001.patch
          0.9 kB
          Hui Fei
        2. HDFS-15062.002.patch
          0.9 kB
          Hui Fei
        3. HDFS-15062.003.patch
          1.0 kB
          Hui Fei

          Activity

            People

            • Assignee:
              ferhui Hui Fei
              Reporter:
              ferhui Hui Fei

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment