Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-4714

Log short messages in Namenode RPC server for exceptions meant for clients

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.7, 2.1.0-beta, 3.0.0-alpha1
    • Fix Version/s: 2.1.0-beta, 0.23.8
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Namenode can slow down significantly if a rogue client/job issues massive number of requests that will fail. E.g. permission denied, quota overage, etc. The major contributing factor in slow down is the long namenode log message, which includes full stack trace.

      Previously similar issues involving safe mode and standby node have been addressed and we can extend it to suppress logging stack traces for configured list of exception classes.

        Attachments

        1. hdfs-4714.patch.txt
          3 kB
          Kihwal Lee
        2. hdfs-4714.branch-2.patch.txt
          3 kB
          Kihwal Lee
        3. hdfs-4714.branch-0.23.patch.txt
          3 kB
          Kihwal Lee

          Issue Links

            Activity

              People

              • Assignee:
                kihwal Kihwal Lee
                Reporter:
                kihwal Kihwal Lee
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: