I'm attaching patch v001. This uses the same parallel test execution techniques that have been effective in hadoop-common and hadoop-hdfs. I updated pom.xml to add the parallel-tests profile and changed some test code to generate testing paths based on the Surefire fork number, so that they're guaranteed to be isolated from each other.
HADOOP-12801 is a pre-requisite for this patch.
There are some limitations to this approach. Tests that cover root directory access, multi-part upload, or YARN usage would run into conflicts on shared resources. To handle this, I've moved those tests to a separate Surefire execution step, which runs sequentially instead of parallelized. A future improvement could be to refactor tests related to root out of FileSystemContractBaseTest, so that the non-root tests in that suite can run in parallel. TestS3AFileSystemContract takes ~90 seconds to run, so it would be nice to parallelize that with other suites. I chose not to step outside hadoop-aws for the scope of the current patch though.
While I was editing pom.xml, I also removed an unused dependency on Jackson. This change is backwards-compatible, because clients who had been relying on receiving Jackson via transitive dependency through hadoop-aws will continue to receive it through the dependency chain hadoop-aws -> hadoop-common -> Jackson.
In my environment, plain mvn test runs in ~30 minutes, and mvn -Pparallel-tests test runs in ~13 minutes.
Steve Loughran, would you please review when you get a chance?