Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-19481

Add support for a flink native GCS FileSystem

    XMLWordPrintableJSON

    Details

      Description

      Currently, GCS is supported but only by using the hadoop connector[1]

       

      The objective of this improvement is to add support for checkpointing to Google Cloud Storage with the Flink File System,

       

      This would allow the `gs://` scheme to be used for savepointing and checkpointing. Long term, it would be nice if we could use the GCS FileSystem as a source and sink in flink jobs as well. 

       

      Long term, I hope that implementing a flink native GCS FileSystem will simplify usage of GCS because the hadoop FileSystem ends up bringing in many unshaded dependencies.

       

      [1] https://github.com/GoogleCloudDataproc/hadoop-connectors

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                baugarten Ben Augarten
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated: