Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-2962

ZKRMStateStore: Limit the number of znodes under a znode

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.9.0, 3.0.0-alpha4
    • Component/s: resourcemanager
    • Labels:
      None

      Description

      We ran into this issue where we were hitting the default ZK server message size configs, primarily because the message had too many znodes even though they individually they were all small.

        Attachments

        1. YARN-2962.006.patch
          62 kB
          Daniel Templeton
        2. YARN-2962.007.patch
          62 kB
          Varun Saxena
        3. YARN-2962.008.patch
          66 kB
          Varun Saxena
        4. YARN-2962.008.patch
          66 kB
          Varun Saxena
        5. YARN-2962.009.patch
          65 kB
          Varun Saxena
        6. YARN-2962.01.patch
          32 kB
          Varun Saxena
        7. YARN-2962.010.patch
          65 kB
          Varun Saxena
        8. YARN-2962.011.patch
          65 kB
          Varun Saxena
        9. YARN-2962.04.patch
          58 kB
          Varun Saxena
        10. YARN-2962.05.patch
          63 kB
          Varun Saxena
        11. YARN-2962.2.patch
          31 kB
          Arun Suresh
        12. YARN-2962.3.patch
          32 kB
          Arun Suresh

          Issue Links

          duplicates

          Bug - A problem which impairs or prevents the functions of the product. YARN-2368 ResourceManager failed when ZKRMStateStore tries to update znode data larger than 1MB

          • Critical - Crashes, loss of data, severe memory leak.
          • Reopened
          relates to

          Bug - A problem which impairs or prevents the functions of the product. YARN-6703 RM startup failure with old state store due to version mismatch

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. YARN-3643 Provide a way to store only running applications in the state store

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. YARN-7262 Add a hierarchy into the ZKRMStateStore for delegation token znodes to prevent jute buffer overflow

          • Major - Major loss of function.
          • Resolved

            Activity

              People

              • Assignee:
                varun_saxena Varun Saxena
                Reporter:
                kasha Karthik Kambatla
              • Votes:
                0 Vote for this issue
                Watchers:
                31 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: