Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-4714

Log short messages in Namenode RPC server for exceptions meant for clients

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.23.7, 2.1.0-beta, 3.0.0-alpha1
    • 2.1.0-beta, 0.23.8
    • namenode
    • None
    • Reviewed

    Description

      Namenode can slow down significantly if a rogue client/job issues massive number of requests that will fail. E.g. permission denied, quota overage, etc. The major contributing factor in slow down is the long namenode log message, which includes full stack trace.

      Previously similar issues involving safe mode and standby node have been addressed and we can extend it to suppress logging stack traces for configured list of exception classes.

      Attachments

        1. hdfs-4714.branch-0.23.patch.txt
          3 kB
          Kihwal Lee
        2. hdfs-4714.branch-2.patch.txt
          3 kB
          Kihwal Lee
        3. hdfs-4714.patch.txt
          3 kB
          Kihwal Lee

        Issue Links

          Activity

            People

              kihwal Kihwal Lee
              kihwal Kihwal Lee
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: