Uploaded image for project: 'Camel'
  1. Camel
  2. CAMEL-16287

camel-aws2-sqs should use pagination for deciding which aws sqs queues it should create

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.25.3, 3.8.0
    • Fix Version/s: 3.10.0
    • Component/s: None
    • Labels:
      None
    • Estimated Complexity:
      Unknown

      Description

      There is an auto create queue feature for aws-sqs in camel-sqs component. Upon camel context creation / route building, it issues an API call to list all sqs queues, so it skips the creation for already existing ones. If camel context has a for example consumer route for an sqs queue that is not listed in the response of this API call, camel will try to to create it.

      However, this API only lists the first 1000 queues in the current account. If there are more then 1000 queues, camel-sqs will try to create a new one, and might fail. In our company's account, we have 1442 queues in our account and Camel tried to create a .fifo queue because it thought it wasnt already existing, resulting in application startup error.

      Relevant code: https://github.com/apache/camel/blob/master/components/camel-aws2-sqs/src/main/java/org/apache/camel/component/aws2/sqs/Sqs2Endpoint.java#L174

      Relevant API docs at AWS: https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/sqs/AmazonSQS.html#listQueues-com.amazonaws.services.sqs.model.ListQueuesRequest-

      Solution is straightforward: use pagination and load all queues.

        Attachments

          Activity

            People

            • Assignee:
              acosentino Andrea Cosentino
              Reporter:
              gyorgy.abraham.dpc Gyorgy Abraham
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: