The new Multipart Uploader API of HDFS-13186 needs to work with S3Guard, with the tests to demonstrate this
- move from low-level calls of S3A client to calls of WriteOperationHelper; adding any new methods are needed there.
- Tests. the tests of
HDFS-13713. - test execution, with -DS3Guard, -DAuth
There isn't an S3A version of AbstractSystemMultipartUploaderTest, and even if there was, it might not show that S3Guard was bypassed, because there's no checks that listFiles/listStatus shows the newly committed files.
Similarly, because MPU requests are initiated in S3AMultipartUploader, encryption settings are't picked up. Files being uploaded this way are not being encrypted
- contains
-
HDFS-13766 HDFS Classes used for implementation of Multipart uploads to move to hadoop-common JAR
-
- Resolved
-
-
HADOOP-16330 Regression: TestStagingPartitionedJobCommit failing with empty etag list
-
- Resolved
-
- depends upon
-
HDFS-13713 Add specification of Multipart Upload API to FS specification, with contract tests
-
- Resolved
-
-
HDFS-13186 [PROVIDED Phase 2] Multipart Uploader API
-
- Resolved
-
- is related to
-
HADOOP-15631 Remove transient dependency on hadoop-hdfs-client
-
- Open
-
-
HADOOP-15604 Bulk commits of S3A MPUs place needless excessive load on S3 & S3Guard
-
- Resolved
-