Description
Expect a PoC patch for this in a couple of days;
- it'll depend on an SDK update to work, plus a couple of of other minor changes
- Adds command line option too
hadoop s3guard select -header use -compression gzip -limit 100 s3a://landsat-pds/scene_list.gz" \ "SELECT s.entityId FROM S3OBJECT s WHERE s.cloudCover = '0.0' "
For wider use we'll need to implement the HADOOP-15229 so that callers can pass down the expression along with any other parameters
Attachments
Attachments
Issue Links
- depends upon
-
HADOOP-15346 S3ARetryPolicy for 400/BadArgument to be "fail"
-
- Resolved
-
-
HADOOP-15229 Add FileSystem builder-based openFile() API to match createFile(); S3A to implement S3 Select through this API.
-
- Resolved
-
-
HADOOP-15606 FSDataInputStream/FSInputStream to implement getFileLength()
-
- Open
-
- is depended upon by
-
MAPREDUCE-7182 MapReduce input format/record readers to support S3 select queries
-
- Resolved
-
- is part of
-
HADOOP-15229 Add FileSystem builder-based openFile() API to match createFile(); S3A to implement S3 Select through this API.
-
- Resolved
-