Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-7386

JBOD threshold to prevent unbalanced disk utilization

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Low
    • Resolution: Fixed
    • 2.0.12, 2.1.3
    • None
    • None

    Description

      Currently the pick the disks are picked first by number of current tasks, then by free space. This helps with performance but can lead to large differences in utilization in some (unlikely but possible) scenarios. Ive seen 55% to 10% and heard reports of 90% to 10% on IRC. With both LCS and STCS (although my suspicion is that STCS makes it worse since harder to be balanced).

      I purpose the algorithm change a little to have some maximum range of utilization where it will pick by free space over load (acknowledging it can be slower). So if a disk A is 30% full and disk B is 5% full it will never pick A over B until it balances out.

      Attachments

        1. 7386-2.0-v3.txt
          16 kB
          Robert Stupp
        2. 7386-2.0-v4.txt
          23 kB
          Robert Stupp
        3. 7386-2.0-v5.txt
          148 kB
          Robert Stupp
        4. 7386-2.1-v3.txt
          18 kB
          Robert Stupp
        5. 7386-2.1-v4.txt
          26 kB
          Robert Stupp
        6. 7386-2.1-v5.txt
          26 kB
          Robert Stupp
        7. 7386-v1.patch
          15 kB
          Robert Stupp
        8. 7386v2.diff
          23 kB
          Robert Stupp
        9. Mappe1.ods
          62 kB
          Robert Stupp
        10. mean-writevalue-7disks.png
          48 kB
          Lyuben Todorov
        11. patch_2_1_branch_proto.diff
          10 kB
          Chris Lohfink
        12. sstable-count-second-run.png
          30 kB
          Lyuben Todorov
        13. test_regression_no_patch.jpg
          84 kB
          Alan Boudreault
        14. test_regression_with_patch.jpg
          94 kB
          Alan Boudreault
        15. test1_no_patch.jpg
          53 kB
          Alan Boudreault
        16. test1_with_patch.jpg
          59 kB
          Alan Boudreault
        17. test2_no_patch.jpg
          65 kB
          Alan Boudreault
        18. test2_with_patch.jpg
          69 kB
          Alan Boudreault
        19. test3_no_patch.jpg
          59 kB
          Alan Boudreault
        20. test3_with_patch.jpg
          73 kB
          Alan Boudreault

        Issue Links

          breaks

          Bug - A problem which impairs or prevents the functions of the product. CASSANDRA-8716 "java.util.concurrent.ExecutionException: java.lang.AssertionError: Memory was freed" when running cleanup

          • Low -
          • Resolved
          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. CASSANDRA-7615 Data isn't written to the disk with enough space while using multiple data_file_directories

          • Normal -
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. CASSANDRA-8571 Free space management does not work very well

          • Normal -
          • Resolved
          is related to

          Improvement - An improvement or enhancement to an existing feature or task. CASSANDRA-8329 LeveledCompactionStrategy should split large files across data directories when compacting

          • Normal -
          • Resolved
          links to

          Web Link Git branch (2.0)

          Web Link Git branch (2.1)

          Activity

            People

              snazy Robert Stupp
              cnlwsu Chris Lohfink
              Robert Stupp
              Yuki Morishita
              Alan Boudreault Alan Boudreault
              Votes:
              5 Vote for this issue
              Watchers:
              18 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: