Details
-
Improvement
-
Status: Closed
-
Not a Priority
-
Resolution: Not A Problem
-
None
-
None
-
None
Description
I wanna use Flink SQL reads S3 bucket data. But now I found it ONLY supports absolute path, which means I can not read all content in the bucket.
My SQL statements write as below:
CREATE TABLE file_data ( a BIGINT, b STRING, c STRING, d DOUBLE, e BOOLEAN, f DATE, g STRING,h STRING, i STRING, j STRING, k STRING, l STRING, m STRING, n STRING, o STRING, p FLOAT ) WITH ( 'connector' = 'filesystem', 'path' = 's3a://my-bucket', 'format' = 'parquet' ); SELECT COUNT(*) FROM file_data;
The exception info:
Caused by: java.lang.IllegalArgumentException: path must be absolute at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88) ~[flink-s3-fs-hadoop-1.13.1.jar:1.13.1] at org.apache.hadoop.fs.s3a.s3guard.PathMetadata.<init>(PathMetadata.java:68) ~[flink-s3-fs-hadoop-1.13.1.jar:1.13.1] at org.apache.hadoop.fs.s3a.s3guard.PathMetadata.<init>(PathMetadata.java:60) ~[flink-s3-fs-hadoop-1.13.1.jar:1.13.1] at org.apache.hadoop.fs.s3a.s3guard.PathMetadata.<init>(PathMetadata.java:56) ~[flink-s3-fs-hadoop-1.13.1.jar:1.13.1] at org.apache.hadoop.fs.s3a.s3guard.S3Guard.putAndReturn(S3Guard.java:149) ~[flink-s3-fs-hadoop-1.13.1.jar:1.13.1]
Is there any solution to meet my requirement ?