Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-7386

JBOD threshold to prevent unbalanced disk utilization

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Low
    • Resolution: Fixed
    • 2.0.12, 2.1.3
    • None
    • None

    Description

      Currently the pick the disks are picked first by number of current tasks, then by free space. This helps with performance but can lead to large differences in utilization in some (unlikely but possible) scenarios. Ive seen 55% to 10% and heard reports of 90% to 10% on IRC. With both LCS and STCS (although my suspicion is that STCS makes it worse since harder to be balanced).

      I purpose the algorithm change a little to have some maximum range of utilization where it will pick by free space over load (acknowledging it can be slower). So if a disk A is 30% full and disk B is 5% full it will never pick A over B until it balances out.

      Attachments

        1. test3_with_patch.jpg
          73 kB
          Alan Boudreault
        2. test3_no_patch.jpg
          59 kB
          Alan Boudreault
        3. test2_with_patch.jpg
          69 kB
          Alan Boudreault
        4. test2_no_patch.jpg
          65 kB
          Alan Boudreault
        5. test1_with_patch.jpg
          59 kB
          Alan Boudreault
        6. test1_no_patch.jpg
          53 kB
          Alan Boudreault
        7. test_regression_with_patch.jpg
          94 kB
          Alan Boudreault
        8. test_regression_no_patch.jpg
          84 kB
          Alan Boudreault
        9. sstable-count-second-run.png
          30 kB
          Lyuben Todorov
        10. patch_2_1_branch_proto.diff
          10 kB
          Chris Lohfink
        11. mean-writevalue-7disks.png
          48 kB
          Lyuben Todorov
        12. Mappe1.ods
          62 kB
          Robert Stupp
        13. 7386v2.diff
          23 kB
          Robert Stupp
        14. 7386-v1.patch
          15 kB
          Robert Stupp
        15. 7386-2.1-v5.txt
          26 kB
          Robert Stupp
        16. 7386-2.1-v4.txt
          26 kB
          Robert Stupp
        17. 7386-2.1-v3.txt
          18 kB
          Robert Stupp
        18. 7386-2.0-v5.txt
          148 kB
          Robert Stupp
        19. 7386-2.0-v4.txt
          23 kB
          Robert Stupp
        20. 7386-2.0-v3.txt
          16 kB
          Robert Stupp

        Issue Links

          breaks

          Bug - A problem which impairs or prevents the functions of the product. CASSANDRA-8716 "java.util.concurrent.ExecutionException: java.lang.AssertionError: Memory was freed" when running cleanup

          • Low -
          • Resolved
          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. CASSANDRA-7615 Data isn't written to the disk with enough space while using multiple data_file_directories

          • Normal -
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. CASSANDRA-8571 Free space management does not work very well

          • Normal -
          • Resolved
          is related to

          Improvement - An improvement or enhancement to an existing feature or task. CASSANDRA-8329 LeveledCompactionStrategy should split large files across data directories when compacting

          • Normal -
          • Resolved
          links to

          Web Link Git branch (2.0)

          Web Link Git branch (2.1)

          Activity

            People

              snazy Robert Stupp
              cnlwsu Chris Lohfink
              Robert Stupp
              Yuki Morishita
              Alan Boudreault Alan Boudreault
              Votes:
              5 Vote for this issue
              Watchers:
              18 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: