[BEAM-6730] Enable configuration of Java transforms (specifically IO) through other SDKs

• Type: New Feature
• Status: Triage Needed
• Priority: P2
• Resolution: Fixed
• Fix Version/s: 2.12.0
• Component/s: runner-flink, sdk-java-core, sdk-py-core
• Labels: Done
• Assignee: Maximilian Michels
• Reporter: Maximilian Michels
• Created: 22/Feb/19 14:24
• Updated: 13/Apr/23 10:55
• Resolved: 02/Apr/19 15:47

Description

Since https://github.com/apache/beam/pull/7316 we can reference external transforms which are transforms only available in a "foreign" SDKs. This allows us to fill the gap in terms of missing transforms in the Python and Go SDK, specifically IO transforms.

We can start collecting/exposing transforms that Beam users need. The following transforms could be interesting:

• KafkaIO / KinesisIO
• CassandraIO / ElasticserchIO / Hbase / Redis
• JDBC
• S3 file system
• GenerateSequence

See also https://s.apache.org/beam-cross-language-io and BEAM-6485.

CC robertwb chamikara thw

Issue Links

• links to: GitHub Pull Request #7875
• links to: GitHub Pull Request #8249
• links to: GitHub Pull Request #8321