[CAMEL-15203] camel-salesforce - Camel looses subscription when Invalid Replay ID is trown by SFDC

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.0, 3.4.0
• Fix Version/s: 3.14.0
• Component/s: camel-salesforce
• Estimated Complexity: Unknown
• Assignee: Unassigned
• Reporter: Martijn Cremer
• Created: 17/Jun/20 12:31
• Updated: 15/Nov/21 04:26
• Resolved: 14/Nov/21 09:03

Description

SFDC has several errors documetned: https://developer.salesforce.com/docs/atlas.en-us.api_streaming.meta/api_streaming/streaming_error_codes.htm

 

There are several problems: How the `SubscriptionHelper` is handling connection errors. At the moment it has a hardcoded 401 and 403 handling. But there are several valid  errors as in the specication above disibed.

 

Atm if we hit these errors, in example the 400 with the message body message {{The replayId {replay_id} you provided was invalid. Please provide a valid ID, -2 to replay all events, or -1 to replay only new events.. }}You would expect that when we get this error, appache camel tries to retrieve it with a replay_id with -2 or -1. At the moment if you get this error you loose subscription total to the channel.

I sugest a generic error handler:

Psudo code:

public class SalesForceStreamingErrorHandler {
    APIversionMandatory("400", "API version in the URI is mandatory"),
    UnsupportedAPIversion("400", "Unsupported API version"),
    InvalidConnectionType("400", " Invalid connection type"),
    ChannelNotExists("400", "The channel you requested to subscribe to does not exis"),
    ChannelNameMissing("400", "Channel name not specified"),
    IncorrectChannelName("400", "Channel subscriptions must start with a leading '/'"),
    QueryFieldsDoNotExist("400", "Query fields {query_fields} do not exist on the topic entity"),
    CookieMissing("400", "Client client_name has established a session, but no cookie_name cookie present"),
    InvalidReplayID("400", "The replayId {replay_id} you provided was invalid. Please provide a valid ID, -2 to replay all events, or -1 to replay only new events."),
    AuthenticationInvalid("401", "Authentication invalid",true,false),
    RequestRequiresAuthentication("401", "Request requires authentication", true,false),
    CannotCreateChannel("403", "Cannot create channel"),
    AccessToTopic("403", "Subscriber does not have access to the entity in this topic"),
    AccessToField("403", "Subscriber does not have access to all fields referenced in the where clause of the PushTopic"),
    HandshakeDenied("403", "Handshake denied"),
    IncompleteHandshake("403", "Client has not completed handshake"),
    OrganisationConcurendUser("403", "Organization concurrent user limit exceeded"),
    OrganisationDailyLimit("403", "Organization total events daily limit exceeded"),
    RestrictedChannel("403", "Restricted channel"),
    UserNotEnabled("403", "User not enabled for streaming"),
    UserNotAllowed("403", "User not allowed to subscribe CDC without View All Data permissions"),
    SubscriptionLimit("403", "Subscription limit exceeded for this topic"),
    UnknownClient("403", "Unknown client"),
    DDOS_PROTECTION("403",
            "To protect all customers from excessive use and Denial of Service attacks, we limit the number of simultaneous connections per server. Your request has been denied because this limit has been exceeded. Please try your request again later."),
    MaxRequstSize("413", "Maximum Request Size Exceeded"),
    ServerBusY("503", "Server is too busy. Please try your request again later.");

With a new configuration option:

// Streaming API properties
@UriParam
private Optional<Long> fallbackReplayId;

With new logic in the SubscriptionHelper to reconnect with the fallback replayID when set. This way we can also improve the eventually the robustness of how error msges are handled in this component. At the moment we have a lot of problems with the replayID bc they can expire afhter 72 hours on the buss and if you had downtime with the bayeux buss, having this missed is a problem.

Issue Links

• relates to: CAMEL-17195 camel-salesforce: Implement fallbackReplayId for streaming API