Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-13768

Update documentation regarding `path style access` for S3 filesystem implementations

    XMLWordPrintableJSON

    Details

      Description

      The documentation related to various properties that can be provided for the s3 sink is not very informative. According to the code in flink-s3-fs-base/flink-s3-fs-hadoop, any property specified as s3.<some-property> is transformed to fs.s3a.<some-property>.

       

      When interacting with s3 compatible file systems such as CEPH/Minio, default configuration properties might not be sufficient. One such property is the fs.s3a.path.style.access. This property enables different modes of access to the s3 buckets. By default if this property is not set, virtual host style addressing is used. The documentation should mention how this property can be passed on as a flink conf property.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                asamudrala Achyuth Narayan Samudrala
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 10m
                  10m