[BEAM-8542] Add async write to AWS SNS IO & remove retry logic

• Type: Improvement
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Fix Version/s: 2.22.0
• Component/s: io-java-aws
• Assignee: Ajo Thomas
• Reporter: Ajo Thomas
• Created: 01/Nov/19 15:19
• Updated: 08/Nov/21 18:45
• Resolved: 05/May/20 10:49

Description

• While working with SNS IO for one of my work-related projects, I found that the IO uses synchronous publishes during writes. I had a simple mock pipeline where I was reading from a kinesis stream and publishing it to SNS using Beam's SNS IO. For comparison, I also had a lamdba which did the same using asynchronous publishes but was about 5x faster. Changing the SNS IO to use async publishes would improve publish latencies.
• SNS IO also has some retry logic which isn't required as SNS clients can handle retries. The retry logic in the SNS client is user-configurable and therefore, an explicit retry logic in SNS IO is not required

I have a working version of the IO with these changes, will create a PR linking this ticket to it once I get some feedback here.

Issue Links

• is related to: BEAM-13203 Potential data loss when using SnsIO.writeAsync
• links to: GitHub Pull Request #10078