You are correct Claus, moving the stop to before the advice makes the issue go away. I had set it up this way because I normally do the advice configuration in the "setup" call for a testing class. I then created a number of different tests/methods within the class. This causes the route to be restarted before each test, thus my call to stop it after the advice. Maybe this isn't the best way to setup my normal tests.