[CAMEL-8539] S3 marker does not work

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 2.15.0
• Component/s: camel-aws
• Estimated Complexity: Novice
• Assignee: Unassigned
• Reporter: Alexey Hanin
• Created: 24/Mar/15 16:23
• Updated: 28/Jan/16 13:09
• Resolved: 28/Jan/16 13:09

Description

S3Consumer doesn't set marker correctly, when deleteAfterRead is set to false. Therefore, on every subsequent poll, it starts from the beginning of the list.

The reason for that is the following code in S3Consumer:

S3Consumer.java

            if (!getConfiguration().isDeleteAfterRead()) {
                // where marker is track
                marker = listObjects.getMarker();
            }

Accordingly to S3 client documentation, getMarker() return a marker previously set in request. Instead, getNextMarker() shall be used. Although, simply changing it to use getNextMarker() introduces an endless loop iterating over the same set of keys under the prefix, because when reached the end of the key set, getNextMarker() returns null, and on the next poll, the head of the list will be returned, and so on.

Issue Links

• duplicates: CAMEL-8431 Consume all files in aws S3 bucket where deleteAfterRead = false