Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.10.1
    • Component/s: container
    • Labels:
      None

      Description

      This ticket tracks work to monitor the disk space used by stores in samza. It is the initial step in a larger effort to restrict excessive disk usage.

      1. baseline1.txt
        1 kB
        Chris Pettitt
      2. baseline2.txt
        1 kB
        Chris Pettitt
      3. experiment.txt
        1 kB
        Chris Pettitt
      4. rb45504.patch
        27 kB
        Chris Pettitt

        Activity

        Show
        cpettitt-linkedin Chris Pettitt added a comment - https://reviews.apache.org/r/45504/
        Hide
        cpettitt-linkedin Chris Pettitt added a comment -

        Performance seems essentially unchanged with this feature enabled and disabled. In our lab, I measured requests processed per second with the feature off (baseline1 and baseline2) and on (experiment) and got the following summary stats:

                    Min. 1st Qu. Median   Mean 3rd Qu.   Max.
        baseline1  20000  196500 210800 199600  225900 237100
        baseline2  10000  196300 210000 198700  223400 240000
        experiment 10000  208200 220300 205300  225300 239000
        

        Interestingly it seems like the throughput occasionally drops on all three (baseline1, baseline2, experiment) and we get nice round numbers (e.g 30,000, 50,000) for a few seconds in a row.

        Show
        cpettitt-linkedin Chris Pettitt added a comment - Performance seems essentially unchanged with this feature enabled and disabled. In our lab, I measured requests processed per second with the feature off (baseline1 and baseline2) and on (experiment) and got the following summary stats: Min. 1st Qu. Median Mean 3rd Qu. Max. baseline1 20000 196500 210800 199600 225900 237100 baseline2 10000 196300 210000 198700 223400 240000 experiment 10000 208200 220300 205300 225300 239000 Interestingly it seems like the throughput occasionally drops on all three (baseline1, baseline2, experiment) and we get nice round numbers (e.g 30,000, 50,000) for a few seconds in a row.
        Hide
        nickpan47 Yi Pan (Data Infrastructure) added a comment -

        Merged and submitted. Thanks!

        Show
        nickpan47 Yi Pan (Data Infrastructure) added a comment - Merged and submitted. Thanks!

          People

          • Assignee:
            cpettitt-linkedin Chris Pettitt
            Reporter:
            cpettitt-linkedin Chris Pettitt
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development