[CASSSIDECAR-140] Updating traffic shaping options throws IllegalStateException

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 1.0
• Component/s: Rest API
• Bug Category: Correctness - Unrecoverable Corruption / Loss
• Severity: Normal
• Complexity: Low Hanging Fruit
• Discovered By: DTest

Source Control Link

https://github.com/apache/cassandra-sidecar/commit/971941cac0ca62d03503509b92d035624388ffba

Test and Documentation Plan

Added a unit test that surfaces the issue. The unit test uses a non-zero port, which will help surface the issue.

• Assignee: Francisco Guerrero
• Reporter: Francisco Guerrero

Authors

Francisco Guerrero

Reviewers

Arjun Ashok, Saranya Krishnakumar, Yifan Cai

• Created: 06/Aug/24 10:59
• Updated: 06/Aug/24 22:06
• Resolved: 06/Aug/24 22:06

Description

When updating the traffic shaping options in Sidecar in org.apache.cassandra.sidecar.server.Server#updateTrafficShapingOptions, we are encountering a bug in vert.x. The problem happens in io.vertx.core.net.impl.TCPServerBase#updateTrafficShapingOptions where the trafficShapingHandler is null for childHandler}}s. When a {{null trafficShapingHandler is encountered, the following exception is thrown:

throw new IllegalStateException("Unable to update traffic shaping options because the server was not configured " +
                                "to use traffic shaping during startup");

I propose a stopgap measure to fix the issue in Sidecar while we wait for a new vert.x release that includes a fix for this issue. Without a fix, we risk leaving Sidecar in unknown state after updating the traffic shaping options. Because applying the traffic shaping options can succeed or fail before encountering the exception. This can potentially leave a cluster of Sidecar servers in an inconsistent state across all Sidecars. The only option to return to a well-known state is by restarting the Sidecar process across the cluster with the updated traffic shaping options applied in the yaml before starting the process.