Details
-
Improvement
-
Status: Resolved
-
Minor
-
Resolution: Fixed
-
None
-
None
-
Unknown
Description
AWS S3 support multipart upload and as today we are using such functionality in the camel-aws3-s3 component to split a message body into chunks of a configurable size and upload them as parts to S3 but it would be nice to have more flexibility as example to be able to treat individual exchanges as chunks.
In fact it is about using S3 as a sort of aggregation repository.
As example, assuming we want to implement a workflow like:
- get records from a database
- filter and transform records
- aggregate records
- send the result to s3
That would require to set-up a local aggregation repository which is ok but increase the resources required to run the integration as well as the moving parts. By leveraging multipart uploads from S3 we could avoid the local aggregation repository.