Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-2962

ZKRMStateStore: Limit the number of znodes under a znode

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • 2.6.0
    • 2.9.0, 3.0.0-alpha4
    • resourcemanager
    • None

    Description

      We ran into this issue where we were hitting the default ZK server message size configs, primarily because the message had too many znodes even though they individually they were all small.

      Attachments

        1. YARN-2962.01.patch
          32 kB
          Varun Saxena
        2. YARN-2962.2.patch
          31 kB
          Arun Suresh
        3. YARN-2962.3.patch
          32 kB
          Arun Suresh
        4. YARN-2962.04.patch
          58 kB
          Varun Saxena
        5. YARN-2962.05.patch
          63 kB
          Varun Saxena
        6. YARN-2962.006.patch
          62 kB
          Daniel Templeton
        7. YARN-2962.007.patch
          62 kB
          Varun Saxena
        8. YARN-2962.008.patch
          66 kB
          Varun Saxena
        9. YARN-2962.008.patch
          66 kB
          Varun Saxena
        10. YARN-2962.009.patch
          65 kB
          Varun Saxena
        11. YARN-2962.010.patch
          65 kB
          Varun Saxena
        12. YARN-2962.011.patch
          65 kB
          Varun Saxena

        Issue Links

          duplicates

          Bug - A problem which impairs or prevents the functions of the product. YARN-2368 ResourceManager failed when ZKRMStateStore tries to update znode data larger than 1MB

          • Critical - Crashes, loss of data, severe memory leak.
          • Reopened
          relates to

          Bug - A problem which impairs or prevents the functions of the product. YARN-6703 RM startup failure with old state store due to version mismatch

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. YARN-3643 Provide a way to store only running applications in the state store

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. YARN-7262 Add a hierarchy into the ZKRMStateStore for delegation token znodes to prevent jute buffer overflow

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              varun_saxena Varun Saxena
              kasha Karthik Kambatla
              Votes:
              0 Vote for this issue
              Watchers:
              32 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: