[HADOOP-19133] "No test bucket" error in ITestS3AContractVectoredRead if provided via CLI property

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 3.4.0, 3.3.6
• Component/s: fs/s3, test
• Assignee: Unassigned
• Reporter: Attila Doroszlai
• Created: 29/Mar/24 19:04
• Updated: 02/Apr/24 09:35

Description

ITestS3AContractVectoredRead fails with NullPointerException: No test bucket if test bucket is defined as -Dtest.fs.s3a.name=... via CLI , not in auth-keys.xml. The same setup works for other S3A contract tests. Tested on 3.3.6.

src/test/resources/auth-keys.xml

<configuration>
  <property>
    <name>fs.s3a.endpoint</name>
    <value>${test.fs.s3a.endpoint}</value>
  </property>
  <property>
    <name>fs.contract.test.fs.s3a</name>
    <value>${test.fs.s3a.name}</value>
  </property>
</configuration>

export AWS_ACCESS_KEY_ID='<redacted>'
export AWS_SECRET_KEY='<redacted>'
mvn -Dtest=ITestS3AContractVectoredRead -Dtest.fs.s3a.name="s3a://mybucket" -Dtest.fs.s3a.endpoint="http://localhost:9878/" clean test

test results

Tests run: 46, Failures: 0, Errors: 8, Skipped: 0, Time elapsed: 7.879 s <<< FAILURE! - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractVectoredRead
testMinSeekAndMaxSizeDefaultValues[Buffer type : direct](org.apache.hadoop.fs.contract.s3a.ITestS3AContractVectoredRead)  Time elapsed: 1.95 s  <<< ERROR!
java.lang.NullPointerException: No test bucket
  at org.apache.hadoop.util.Preconditions.checkNotNull(Preconditions.java:88)
  at org.apache.hadoop.fs.s3a.S3ATestUtils.getTestBucketName(S3ATestUtils.java:714)
  at org.apache.hadoop.fs.s3a.S3ATestUtils.removeBaseAndBucketOverrides(S3ATestUtils.java:775)
  at org.apache.hadoop.fs.contract.s3a.ITestS3AContractVectoredRead.testMinSeekAndMaxSizeDefaultValues(ITestS3AContractVectoredRead.java:104)
  ...

testMinSeekAndMaxSizeConfigsPropagation[Buffer type : direct](org.apache.hadoop.fs.contract.s3a.ITestS3AContractVectoredRead)  Time elapsed: 0.176 s  <<< ERROR!
testMultiVectoredReadStatsCollection[Buffer type : direct](org.apache.hadoop.fs.contract.s3a.ITestS3AContractVectoredRead)  Time elapsed: 0.179 s  <<< ERROR!
testNormalReadVsVectoredReadStatsCollection[Buffer type : direct](org.apache.hadoop.fs.contract.s3a.ITestS3AContractVectoredRead)  Time elapsed: 0.155 s  <<< ERROR!
testMinSeekAndMaxSizeDefaultValues[Buffer type : array](org.apache.hadoop.fs.contract.s3a.ITestS3AContractVectoredRead)  Time elapsed: 0.116 s  <<< ERROR!
testMinSeekAndMaxSizeConfigsPropagation[Buffer type : array](org.apache.hadoop.fs.contract.s3a.ITestS3AContractVectoredRead)  Time elapsed: 0.102 s  <<< ERROR!
testMultiVectoredReadStatsCollection[Buffer type : array](org.apache.hadoop.fs.contract.s3a.ITestS3AContractVectoredRead)  Time elapsed: 0.105 s  <<< ERROR!
testNormalReadVsVectoredReadStatsCollection[Buffer type : array](org.apache.hadoop.fs.contract.s3a.ITestS3AContractVectoredRead)  Time elapsed: 0.107 s  <<< ERROR!