Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12516

Suppress the fsnamesystem lock warning on nn startup

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.1.0
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Whenever FsNameSystemLock is held for more than configured value of dfs.namenode.write-lock-reporting-threshold-ms, we log stacktrace and an entry in metrics. Loading FSImage from disk will usually cross this threshold. We can suppress this FsNamesystem lock warning on NameNode startup.

      17/09/20 21:41:39 INFO namenode.FSNamesystem: FSNamesystem write lock held for 7159 ms via
      java.lang.Thread.getStackTrace(Thread.java:1552)
      org.apache.hadoop.util.StringUtils.getStackTrace(StringUtils.java:945)
      org.apache.hadoop.hdfs.server.namenode.FSNamesystem.writeUnlock(FSNamesystem.java:1659)
      org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1074)
      org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:703)
      org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:688)
      org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:752)
      org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:992)
      org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:976)
      org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1701)
      org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1769)
              Number of suppressed write-lock reports: 0
              Longest write-lock held interval: 7159
      

        Attachments

        1. HDFS-12516.01.patch
          6 kB
          Ajay Kumar
        2. HDFS-12516.02.patch
          6 kB
          Ajay Kumar
        3. HDFS-12516.03.patch
          6 kB
          Ajay Kumar

          Activity

            People

            • Assignee:
              ajayydv Ajay Kumar
              Reporter:
              ajayydv Ajay Kumar
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: