Details
-
New Feature
-
Status: Resolved
-
P3
-
Resolution: Fixed
-
None
-
None
Description
Note that this is for providing direct integration with S3 as an Apache Beam FileSystem.
There is already support for using the Hadoop S3 connector by depending on the Hadoop File System module[1], configuring HadoopFileSystemOptions[2] with a S3 configuration[3].
1: https://github.com/apache/beam/tree/master/sdks/java/io/hadoop-file-system
2: https://github.com/apache/beam/blob/master/sdks/java/io/hadoop-file-system/src/main/java/org/apache/beam/sdk/io/hdfs/HadoopFileSystemOptions.java#L53
3: https://wiki.apache.org/hadoop/AmazonS3