Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-2962

ZKRMStateStore: Limit the number of znodes under a znode

VotersStop watchingWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • 2.6.0
    • 2.9.0, 3.0.0-alpha4
    • resourcemanager
    • None

    Description

      We ran into this issue where we were hitting the default ZK server message size configs, primarily because the message had too many znodes even though they individually they were all small.

      Attachments

        1. YARN-2962.011.patch
          65 kB
          Varun Saxena
        2. YARN-2962.010.patch
          65 kB
          Varun Saxena
        3. YARN-2962.009.patch
          65 kB
          Varun Saxena
        4. YARN-2962.008.patch
          66 kB
          Varun Saxena
        5. YARN-2962.008.patch
          66 kB
          Varun Saxena
        6. YARN-2962.007.patch
          62 kB
          Varun Saxena
        7. YARN-2962.006.patch
          62 kB
          Daniel Templeton
        8. YARN-2962.05.patch
          63 kB
          Varun Saxena
        9. YARN-2962.04.patch
          58 kB
          Varun Saxena
        10. YARN-2962.3.patch
          32 kB
          Arun Suresh
        11. YARN-2962.2.patch
          31 kB
          Arun Suresh
        12. YARN-2962.01.patch
          32 kB
          Varun Saxena

        Issue Links

        duplicates

        Bug - A problem which impairs or prevents the functions of the product. YARN-2368 ResourceManager failed when ZKRMStateStore tries to update znode data larger than 1MB

        • Critical - Crashes, loss of data, severe memory leak.
        • Reopened
        relates to

        Bug - A problem which impairs or prevents the functions of the product. YARN-6703 RM startup failure with old state store due to version mismatch

        • Critical - Crashes, loss of data, severe memory leak.
        • Resolved

        Improvement - An improvement or enhancement to an existing feature or task. YARN-3643 Provide a way to store only running applications in the state store

        • Major - Major loss of function.
        • Resolved

        Improvement - An improvement or enhancement to an existing feature or task. YARN-7262 Add a hierarchy into the ZKRMStateStore for delegation token znodes to prevent jute buffer overflow

        • Major - Major loss of function.
        • Resolved

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            varun_saxena Varun Saxena
            kasha Karthik Kambatla
            Votes:
            0 Vote for this issue
            Watchers:
            31 Stop watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment