Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-6825

RM quit due to ApplicationStateData exceed the limit size of znode in zk

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • resourcemanager
    • None

    Description

      YARN-5006 fixes this issue by strict validation for ApplicationStateData length against 1MB(default max jute buffer) during application submission only. There is possibility of thrashing as dead zone was not properly defined/taken care.

      But it do not consider scenarios where ApplicationStateData can be increased later point of time i.e

      1. If app is submitted with less than 1MB during submission, later updated like queue name or life time value or priority is changed. The app update call will be sent to statestore which cause same issue because ApplicationStateData length has increased.
      2. Consider there is no app update, but final state are stored in ZK. This adds up several fields such finishTime, finalState, finalApplicationState. This increases size of ApplicationStateData.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              rohithsharma Rohith Sharma K S
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated: