Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-10781

The Thread of the NM aggregate log is exhausted and no other Application can aggregate the log

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.9.2, 3.3.0
    • Fix Version/s: None
    • Component/s: yarn
    • Labels:
      None

      Description

      We observed more than 100 applications running on one NM.Most of these applications are SparkStreaming applications, but these applications do not have running Containers.When the offline application running on it finishes, the log cannot be reported to HDFS. When we killed a large number of SparkStreaming applications, we found that a large number of log files were being created on the NN side, causing the read and write performance on the NN side to degrade significantly.Causes the business application to time out。

        Attachments

        1. containers.png
          83 kB
          Xiping Zhang
        2. containers.png
          82 kB
          Xiping Zhang
        3. applications.png
          82 kB
          Xiping Zhang

          Activity

          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

              Dates

              • Created:
                Updated:

                Issue deployment