Uploaded image for project: 'Camel'
  1. Camel
  2. CAMEL-16287

camel-aws2-sqs should use pagination for deciding which aws sqs queues it should create

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.25.3, 3.8.0
    • 3.14.4, 3.18.0
    • None
    • None
    • Unknown

    Description

      There is an auto create queue feature for aws-sqs in camel-sqs component. Upon camel context creation / route building, it issues an API call to list all sqs queues, so it skips the creation for already existing ones. If camel context has a for example consumer route for an sqs queue that is not listed in the response of this API call, camel will try to to create it.

      However, this API only lists the first 1000 queues in the current account. If there are more then 1000 queues, camel-sqs will try to create a new one, and might fail. In our company's account, we have 1442 queues in our account and Camel tried to create a .fifo queue because it thought it wasnt already existing, resulting in application startup error.

      Relevant code: https://github.com/apache/camel/blob/master/components/camel-aws2-sqs/src/main/java/org/apache/camel/component/aws2/sqs/Sqs2Endpoint.java#L174

      Relevant API docs at AWS: https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/sqs/AmazonSQS.html#listQueues-com.amazonaws.services.sqs.model.ListQueuesRequest-

      Solution is straightforward: use pagination and load all queues.

      Attachments

        Activity

          People

            acosentino Andrea Cosentino
            gyorgy.abraham.dpc Gyorgy Abraham
            Votes:
            1 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: