[BEAM-9627] KafkaIO needs better support for SSL

• Type: Improvement
• Status: Resolved
• Priority: P3
• Resolution: Not A Problem
• Fix Version/s: Not applicable
• Component/s: io-java-kafka
• Assignee: Unassigned
• Reporter: Daniel Mills
• Created: 28/Mar/20 00:27
• Updated: 26/Oct/20 14:51
• Resolved: 02/Apr/20 12:49

Description

Configuring SSL for kafka requires pointing an option at local files containing keys and roots of trust as described here: https://www.confluent.io/blog/apache-kafka-security-authorization-authentication-encryption/

Currently, it is somewhat tricky to ensure that these files are written before KafkaIO starts reading from the source; one potential option would be to add an init hook where the user could download keys from the keystore of their choice and write them to local files.