[CAMEL-17110] Camel-Kamelets: While using AWS S3 source noticed files were deleted before being consumed at all

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.11.3
• Fix Version/s: 4.x
• Component/s: camel-aws
• Estimated Complexity: Unknown
• Assignee: Andrea Cosentino
• Reporter: Andrea Cosentino
• Created: 20/Oct/21 08:43
• Updated: 09/Aug/23 09:29

Description

- route:
    from:
      uri: "kamelet:aws-s3-source"
      parameters:
        bucketNameOrArn: "camel-kafka-connector"
        accessKey: "access"
        secretKey: "secret"
        region: "eu-west-1"
    steps:
      - to: 
          uri: "kamelet:kafka-not-secured-sink"
          parameters:
            brokers: "localhost:9092"
            topic: "s3-source-topic"

In the log, if you enabled trace you may notice something like:

10:14:24.476 [Camel (AWS-S3-To-Kafka) thread #11 - KafkaProducer[s3-source-topic]] TRACE o.a.c.c.aws2.s3.AWS2S3Consumer - Deleted object from bucket camel-kafka-connector with key jkXzIEbaYyKMTMwGpNHL.txt...

10:14:24.491 [Camel (AWS-S3-To-Kafka) thread #0 - aws2-s3://camel-kafka-connector] TRACE s.a.awssdk.auth.signer.Aws4Signer - AWS4 Canonical Request: GET
/jkXzIEbaYyKMTMwGpNHL.txt

10:14:24.582 [Camel (AWS-S3-To-Kafka) thread #0 - aws2-s3://camel-kafka-connector] DEBUG software.amazon.awssdk.request - Received error response: software.amazon.awssdk.services.s3.model.NoSuchKeyException: The specified key does not exist. 

While, the get should happens before the deletions. This is happening only when using Kamelets. It looks like the exchange completed before the get operation has been done.

Issue Links

• is related to: CAMEL-17111 Camel-AWS2-S3: Supporting Consumer idempotency in some way