Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Fix Version/s: 0.7 beta 3
    • Component/s: Core
    • Labels:
      None

      Description

      with CASSANDRA-1314, we'd like to have dynamic snitch route around slow/failing nodes but not messing with our "pinning" of ranges to nodes unless a threshold of badness is crossed. (Threshold should be optional since other snitches pure dynamic corresponds exactly to what we want.)

      1. 1519.txt
        7 kB
        Brandon Williams

        Activity

        Hide
        Hudson added a comment -

        Integrated in Cassandra #565 (See https://hudson.apache.org/hudson/job/Cassandra/565/)
        Add badness threshold to the dynamic snitch for replica 'pinning'. Patch by brandonwilliams, reviewed by jbellis for CASSANDRA-1519

        Show
        Hudson added a comment - Integrated in Cassandra #565 (See https://hudson.apache.org/hudson/job/Cassandra/565/ ) Add badness threshold to the dynamic snitch for replica 'pinning'. Patch by brandonwilliams, reviewed by jbellis for CASSANDRA-1519
        Hide
        Brandon Williams added a comment -

        Committed.

        Show
        Brandon Williams added a comment - Committed.
        Hide
        Jonathan Ellis added a comment -

        you're right. +1

        Show
        Jonathan Ellis added a comment - you're right. +1
        Hide
        Brandon Williams added a comment -

        1 would mean "use another replica when the pinned one is 100% worse than another". Zero means "use another replica when the pinned one is any amount worse than another", which we optimize to be the original behavior of "sort by score." I suppose we can allow values > 1, though I'm not sure how much sense it would make to use such a large value since phi isn't scaling linearly.

        Show
        Brandon Williams added a comment - 1 would mean "use another replica when the pinned one is 100% worse than another". Zero means "use another replica when the pinned one is any amount worse than another", which we optimize to be the original behavior of "sort by score." I suppose we can allow values > 1, though I'm not sure how much sense it would make to use such a large value since phi isn't scaling linearly.
        Hide
        Jonathan Ellis added a comment -

        since it's a ratio then it needs to be > 1 to actually do something right? comments should reflect that, probably 1.0 should be the default = off.

        Show
        Jonathan Ellis added a comment - since it's a ratio then it needs to be > 1 to actually do something right? comments should reflect that, probably 1.0 should be the default = off.
        Hide
        Brandon Williams added a comment -

        Updated patch fixes the nits, and changes the logic to consult the subsnitch for order if needed.

        Show
        Brandon Williams added a comment - Updated patch fixes the nits, and changes the logic to consult the subsnitch for order if needed.
        Hide
        Jonathan Ellis added a comment -

        after a closer look at the DES code I don't think it matches what the comments in the config file say. I think we need to actually pull the subsnitch value out so we know which one it's supposed to have affinity to, then see if the affinity'd score is over threshold larger than the other, and if so, override it.

        Show
        Jonathan Ellis added a comment - after a closer look at the DES code I don't think it matches what the comments in the config file say. I think we need to actually pull the subsnitch value out so we know which one it's supposed to have affinity to, then see if the affinity'd score is over threshold larger than the other, and if so, override it.
        Hide
        Jonathan Ellis added a comment -

        +1

        minor nits:

        surely an int is plenty large enough?
        first two settings are inlined into DynamicEndpointSnitch but 3rd is not

        Show
        Jonathan Ellis added a comment - +1 minor nits: surely an int is plenty large enough? first two settings are inlined into DynamicEndpointSnitch but 3rd is not
        Hide
        Brandon Williams added a comment -

        Updated patch which uses the value of the badness threshold to control which behavior is used.

        Show
        Brandon Williams added a comment - Updated patch which uses the value of the badness threshold to control which behavior is used.
        Hide
        Brandon Williams added a comment - - edited

        It's more computationally intensive to do the badness calculation all the time, but I guess we can just check if it's greater than zero and eliminate the pinning.

        Show
        Brandon Williams added a comment - - edited It's more computationally intensive to do the badness calculation all the time, but I guess we can just check if it's greater than zero and eliminate the pinning.
        Hide
        Jonathan Ellis added a comment -

        i like the badness threshold solution.

        is the prefer_pinning variable redundant? looks like badness threshold of 0 is equivalent to off.

        Show
        Jonathan Ellis added a comment - i like the badness threshold solution. is the prefer_pinning variable redundant? looks like badness threshold of 0 is equivalent to off.
        Hide
        Brandon Williams added a comment -

        Patch to add a badness threshold and expose a few more tunables.

        Show
        Brandon Williams added a comment - Patch to add a badness threshold and expose a few more tunables.

          People

          • Assignee:
            Brandon Williams
            Reporter:
            Jonathan Ellis
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development