I was not disappointed in your decision but only in the lack of discussion. In each one of the exchanges above we appeared to reach consensus and then suddenly you changed your mind based on conversations offline. I appreciate the chance to resolve this with input from others.
The overlap in functionality doesn't concern me. There are often different ways to accomplish things in Camel and other frameworks. For example, the timer and quartz components offer similar functionality and yet both exist. I would argue that the SQS and SNS components are complementary as opposed to mere duplication.
I don't understand your comment about a consumer not making sense or not being supported by the Amazon API. First, it makes sense because without consumers, there is no reason to ever publish a message. Second, it is supported by the Amazon API because the Amazon API allows you to create a subscription. The subscription endpoint receives the notification messages and by any definition is a consumer.
I intentionally did not support the http, http(s), or smtp protocols because I didn't need them for my use case and in fact don't think I would ever use them in a production environment. Most of my Camel deployments live behind a corporate firewall and as such it is difficult to provide an externally available endpoint. I don't have any experience with the SNS platform in production so I don't know how popular the different subscription types are. As an enterprise user, the SQS subscription type that allowed me to poll for messages seemed good enough. If there was a call from other users to support other subscription types, then I'm sure it could be implemented if necessary although this would greatly expand the dependencies and scope of the component. That said, I've seen done this before for a proprietary WS-Notification/WS-Eventing Camel component that leveraged Jetty for the subscription endpoint.
I don't understand your comment about putting the subscription control into the SNS Producer. It seems weird that you'd create a subscription or check for its creation each time the producer executed.
The simplest use case I can think of that shows the merit of having an SNS consumer is wanting to have a Camel route where the subscription to the topic is temporary. When the route starts, data flows from the topic into the route as messages become available. When the route stops, the flow of data stops. I don't see how you could support this use case.
Of course, there are other use cases that may or may not involve using the Amazon Console to create subscriptions out of band. As I see it, the SNS consumer as it exists in the original submission supports all styles.
- sns consumer with temporary queue
- sns consumer with permanent queue (by name or ARN)
- mix of sqs consumer and AWS console
You're proposing to only support the last in the above list. I think the others have value.
Finally, I think it makes sense from an end user perspective to try and abstract the subscription details away from a user if possible. I don't really care how I receive the messages from the topic, just as long as I get them and that they're verified as coming from SNS. The SNS consumer provides this functionality by bundling the creation of the subscription and the temporary queue in a single endpoint URI. If you separate them, then you're requiring additional configuration from the end user in terms of the subscription and message processing. The user will then need to configure their routes in advance with the additional SQS endpoint and then possibly provide a custom processor on the route to verify the headers for each message. Again, this code is trivial, but why make them go to the trouble when it could all be done for them?