Description
Overview
A key in the S3 bucket cannot access without authentication, even though each bucket/volume allows anonymous reading and listing in its ACLs.
Configuraiton
Create a bucket in a volume, make it accessible from S3, and then put the ACL anonymous::rl to them.
# create a bucket accessible via S3 and put a key ozone sh bucket create /volume/bucket-for-anonymous ozone sh bucket link /volume/bucket-for-anonymous /s3v/bucket-for-anonymous aws s3 --endpoint ... cp README s3://bucket-for-anonymous # set ACLs for anonymous access to the source/s3v buckets, the source/s3v volumes and the key ozone sh bucket addacl volume/bucket-for-anonymous -a anonymous::rl ozone sh bucket addacl s3v/bucket-for-anonymous -a anonymous::rl ozone sh volume addacl volume -a anonymous::rl ozone sh volume addacl s3v -a anonymous::rl # set ACL for the key ozone sh key addacl volume/bucket-for-anonymous/README -a anonymous::r
Case: Access without authentication using wget will fail with 403
Attempting to access to the key, but it fails with 403.
% wget -qO https://HOST/bucket-for-anonymous/README -S
HTTP/1.1 403 Forbidden
Date: Mon, 13 Feb 2023 07:55:58 GMT
Cache-Control: no-cache
Expires: Mon, 13 Feb 2023 07:55:58 GMT
Pragma: no-cache
Content-Type: text/plain
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
X-FRAME-OPTIONS: SAMEORIGIN
Server: Ozone
x-amz-id-2: gT8na4osJZlG
x-amz-request-id: c139bbcf-3d93-4f4f-a6a2-43f75bc0de83
Content-Length: 187
S3G outputs an error message: "Malformed s3 header" as a DEBUG-level message from OzoneClientProducer. This situation means that S3G rejects the access at S3 secrets validation checks.
2023-02-13 15:00:05,079 [qtp731829978-166] DEBUG org.eclipse.jetty.servlet.ServletHandler: chain=Chain@68772dce(NoCacheFilter==org.apache.hadoop.hdds.server.http.NoCacheFilter@740d2e78{inst=true,async=true,src=EMBEDDED:null})->Chain@286a9870(safety==org.apache.hadoop.hdds.server.http.HttpServer2$QuotingInputFilter@6aa3a905{inst=true,async=true,src=EMBEDDED:null})->Chain@2232456a(optional-content-type==org.apache.hadoop.ozone.s3.EmptyContentTypeFilter@d4ab71a{inst=true,async=false,src=DESCRIPTOR:file:///tmp/jetty-0_0_0_0-9879-ozone-s3gateway-1_2_9995006_jar-_-any-17931241039680298355/webapp/WEB-INF/web.xml})->Chain@2629e2cc(info-page-redirect==org.apache.hadoop.ozone.s3.RootPageDisplayFilter@3b4ef7{inst=true,async=false,src=DESCRIPTOR:file:///tmp/jetty-0_0_0_0-9879-ozone-s3gateway-1_2_9995006_jar-_-any-17931241039680298355/webapp/WEB-INF/web.xml})->ChainEnd@7761a29a(jaxrs==org.glassfish.jersey.servlet.ServletContainer@603a422{jsp=null,order=1,inst=true,async=false,src=DESCRIPTOR:file:///tmp/jetty-0_0_0_0-9879-ozone-s3gateway-1_2_9995006_jar-_-any-17931241039680298355/webapp/WEB-INF/web.xml,STARTED}) 2023-02-13 15:00:05,085 [qtp731829978-166] DEBUG org.apache.hadoop.ozone.s3.OzoneClientProducer: Malformed s3 header. awsAccessID: 2023-02-13 15:00:05,314 [qtp731829978-166] DEBUG org.apache.hadoop.ozone.s3.OzoneClientProducer: Error during Client Creation: 2023-02-13 15:00:05,378 [qtp731829978-166] DEBUG org.apache.hadoop.ozone.s3.exception.OS3Exception: toXml val is <Error> 2023-02-13 15:00:05,392 [qtp731829978-166] DEBUG org.eclipse.jetty.server.HttpOutput: write(array HeapByteBuffer@5fe2ddf1[p=0,l=187,c=8192,r=187]={<<<<?xml version="1.0" encod... <RequestId/>\n</Error>\n>>>\x00\x00\x00\x00\x00\x00\x00\x00\x00...\x00\x00\x00\x00\x00\x00\x00}) 2023-02-13 15:00:05,392 [qtp731829978-166] DEBUG org.eclipse.jetty.server.HttpOutput: write(array) s=CLOSING,api=BLOCKED,sc=false,e=null last=true agg=false flush=true async=false, len=187 null 2023-02-13 15:00:05,392 [qtp731829978-166] DEBUG org.eclipse.jetty.server.HttpChannel: sendResponse info=null content=HeapByteBuffer@354f401a[p=0,l=187,c=8192,r=187]={<<<<?xml version="1.0" encod... <RequestId/>\n</Error>\n>>>\x00\x00\x00\x00\x00\x00\x00\x00\x00...\x00\x00\x00\x00\x00\x00\x00} complete=true committing=true callback=Blocker@56ca79aa{null} 2023-02-13 15:00:05,394 [qtp731829978-166] DEBUG org.eclipse.jetty.server.HttpChannel: COMMIT for /bucket-for-anonymous/README on HttpChannelOverHttp@43f236bf{s=HttpChannelState@340e7dcf{s=HANDLING rs=BLOCKING os=COMMITTED is=IDLE awp=false se=false i=true al=0},r=1,c=false/false,a=HANDLING,uri=https://HOST/bucket-for-anonymous/README,age=321} 2023-02-13 15:00:05,394 [qtp731829978-166] DEBUG org.eclipse.jetty.server.HttpConnection: generate: NEED_HEADER for SendCallback@322df2c9[PROCESSING][i=HTTP/1.1{s=403,h=12,cl=187},cb=org.eclipse.jetty.server.HttpChannel$SendCallback@7b69ac28] (null,[p=0,l=187,c=8192,r=187],true)@START 2023-02-13 15:00:05,394 [qtp731829978-166] DEBUG org.eclipse.jetty.http.HttpGenerator: generateHeaders HTTP/1.1{s=403,h=12,cl=187} last=true content=HeapByteBuffer@354f401a[p=0,l=187,c=8192,r=187]={<<<<?xml version="1.0" encod... <RequestId/>\n</Error>\n>>>\x00\x00\x00\x00\x00\x00\x00\x00\x00...\x00\x00\x00\x00\x00\x00\x00}
One possible solution is relaxing S3 secrets validation when ACL has the anonymous scope. So requires fetching ACLs before processing S3 secrets at S3G-side or offloading S3 token validation to OM.