Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Won't Fix
-
None
-
None
-
None
Description
Our swift filesystem is closely aligned with the S3 filesystems in that it follows the same packaging scheme as flink-fs-hadoop-shaded (used by the S3 filesystems), and thus ideally is kept in sync with these filesystems, for example in regards to dependencies.
As it stands, this is not the case. While the S3 modules work against Hadoop 3, swift is still sitting on 2.8.1 . Presumable this difference is due to us having no infrastructure for actually testing our swift filesystem.
Unfortunately it appears that Switft is rather annoying to setup, but I was able find a project that tries to provide a docker image containing everything you need. Maybe we could use this to bootstrap some end-to-end-test, allowing us to finally touch this module again without fear of completely breaking it.
Attachments
Issue Links
- is superceded by
-
-
- Closed
-