Major Description
When I run the IContract* unit tests from s3a adapter to test our s3 gateway I found that some of the unit tests (such as ITestS3AContractGetFileStatus.testListStatusEmptyDirectory) fails with org.apache.hadoop.fs.s3a.AWSBadRequestException.
Checking the request/response I found that the multi delete request (POST to the bucket address with ending with ?delete) fails with HTTP 400 and without any error in the logs.
The multi delete endpoint was tested with the robot tests and it worked well with aws s3 cli:
aws s3api --endpoint http://localhost:9878 delete-objects --bucket buckettest --delete 'Objects=[{Key=multidelete/f1},{Key=multidelete/f2},{Key=multidelete/f4}]'
After some more investigation I found that the XML namespace was missing from the request:
The request was something like this:
<Delete><Object><Key>multidelete/f1</Key></Object><Object><Key>multidelete/f2</Key></Object><Object><Key>multidelete/f4</Key></Object></Delete>
Instead of this:
<Delete xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Object><Key>multidelete/f1</Key></Object><Object><Key>multidelete/f2</Key></Object><Object><Key>multidelete/f4</Key></Object></Delete>
I believe the right approach is to add the namespace to request as it's documented by the AWS documentation but it seems s3a adapter doesn't do it and s3 accepts it.
We need a custom jaxb unmarshaller logic for the multi delete requests to accept requests both with and without namespaces.