Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5734 OrgQueue for easy CapacityScheduler queue configuration management
  3. YARN-7497

Add file system based scheduler configuration store

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.2.0
    • yarn
    • None
    • Reviewed

    Description

      YARN-5947 add LeveldbConfigurationStore using Leveldb as backing store, but it does not support Yarn RM HA.
      YARN-6840 supports RM HA, but too many scheduler configurations may exceed znode limit, for example 10 thousand queues.
      HDFSSchedulerConfigurationStore store conf file in HDFS, when RM failover, new active RM can load scheduler configuration from HDFS.

      Attachments

        1. YARN-7497.001.patch
          7 kB
          Jiandan Yang
        2. YARN-7497.002.patch
          23 kB
          Jiandan Yang
        3. YARN-7497.003.patch
          36 kB
          Jiandan Yang
        4. YARN-7497.004.patch
          36 kB
          Jiandan Yang
        5. YARN-7497.005.patch
          36 kB
          Jiandan Yang
        6. YARN-7497.006.patch
          37 kB
          Jiandan Yang
        7. YARN-7497.007.patch
          9 kB
          Jiandan Yang
        8. YARN-7497.008.patch
          25 kB
          Jiandan Yang
        9. YARN-7497.009.patch
          26 kB
          Jiandan Yang
        10. YARN-7497.010.patch
          28 kB
          Jiandan Yang
        11. YARN-7497.011.patch
          28 kB
          Jiandan Yang
        12. YARN-7497.012.patch
          28 kB
          Weiwei Yang

        Activity

          People

            yangjiandan Jiandan Yang
            yangjiandan Jiandan Yang
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: