Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12943 Consistent Reads from Standby Node
  3. HDFS-13122

Tailing edits should not update quota counts on ObserverNode

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: hdfs, namenode
    • Labels:
      None

      Description

      Currently in FSImage#loadEdits(), after applying a set of edits, we call

      updateCountForQuota(target.getBlockManager().getStoragePolicySuite(), target.dir.rootDir);
      

      to update the quota counts for the entire namespace, which can be very expensive. This makes sense if we are about to become the ANN, since we need valid quotas, but not on an ObserverNode which does not need to enforce quotas.

      This is related to increasing the frequency with which the SbNN can tail edits from the ANN to decrease the lag time for transactions to appear on the Observer.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                xkrogen Erik Krogen
                Reporter:
                xkrogen Erik Krogen
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: