[HADOOP-18545] hadoop-azure integration tests are failing on the trunk branch against non-HNS stores - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Open
Priority: Major
Resolution: Unresolved
Affects Version/s: 3.4.0
Fix Version/s: None
Component/s: fs/azure, test
Labels:
None

Description

I am getting errors when running the hadoop-azure integration test from the trunk branch (commit 0ef572abed624011ad9a9e69d725774d6d00b75b).

This is running on an Azure instance, with HNS disabled and using a Shared Key (NonHNS-SharedKey).

Here are the logs when running those tests:

[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.fs.azurebfs.services.TestAbfsHttpOperation
[INFO] Running org.apache.hadoop.fs.azurebfs.services.TestAbfsClient
[INFO] Running org.apache.hadoop.fs.azurebfs.services.TestAbfsInputStream
[INFO] Running org.apache.hadoop.fs.azurebfs.services.TestAzureADAuthenticator
[INFO] Running org.apache.hadoop.fs.azurebfs.services.TestShellDecryptionKeyProvider
[INFO] Running org.apache.hadoop.fs.azurebfs.services.TestAbfsOutputStream
[INFO] Running org.apache.hadoop.fs.azurebfs.TestAccountConfiguration
[INFO] Running org.apache.hadoop.fs.azurebfs.services.TestAbfsClientThrottlingAnalyzer
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.976 s - in org.apache.hadoop.fs.azurebfs.services.TestAbfsHttpOperation
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.174 s - in org.apache.hadoop.fs.azurebfs.services.TestShellDecryptionKeyProvider
[INFO] Running org.apache.hadoop.fs.azurebfs.services.TestAbfsPerfTracker
[ERROR] Tests run: 9, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 2.921 s <<< FAILURE! - in org.apache.hadoop.fs.azurebfs.TestAccountConfiguration
[ERROR] testConfigPropNotFound(org.apache.hadoop.fs.azurebfs.TestAccountConfiguration)  Time elapsed: 0.102 s  <<< FAILURE!
java.lang.AssertionError: Expected a org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException to be thrown, but got the result: : "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider"
        at org.apache.hadoop.test.LambdaTestUtils.intercept(LambdaTestUtils.java:499)
        at org.apache.hadoop.test.LambdaTestUtils.intercept(LambdaTestUtils.java:384)
        at org.apache.hadoop.fs.azurebfs.TestAccountConfiguration.testMissingConfigKey(TestAccountConfiguration.java:399)
        at org.apache.hadoop.fs.azurebfs.TestAccountConfiguration.testConfigPropNotFound(TestAccountConfiguration.java:386)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
        at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
        at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
        at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
        at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
        at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
        at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
        at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
        at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
        at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
        at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
        at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
        at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
        at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
        at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
        at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
        at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
        at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
        at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
        at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)

[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.396 s - in org.apache.hadoop.fs.azurebfs.services.TestAbfsClient
[INFO] Running org.apache.hadoop.fs.azurebfs.services.TestTextFileBasedIdentityHandler
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.043 s - in org.apache.hadoop.fs.azurebfs.services.TestTextFileBasedIdentityHandler
[INFO] Running org.apache.hadoop.fs.azurebfs.services.TestQueryParams
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.2 s - in org.apache.hadoop.fs.azurebfs.services.TestQueryParams
[INFO] Running org.apache.hadoop.fs.azurebfs.services.TestAbfsRenameRetryRecovery
[INFO] Running org.apache.hadoop.fs.azurebfs.services.TestExponentialRetryPolicy
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.088 s - in org.apache.hadoop.fs.azurebfs.services.TestAzureADAuthenticator
[INFO] Running org.apache.hadoop.fs.azurebfs.TestAbfsInputStreamStatistics
[INFO] Running org.apache.hadoop.fs.azurebfs.utils.TestCachedSASToken
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.651 s - in org.apache.hadoop.fs.azurebfs.utils.TestCachedSASToken
[INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.133 s - in org.apache.hadoop.fs.azurebfs.services.TestAbfsPerfTracker
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.278 s - in org.apache.hadoop.fs.azurebfs.services.TestExponentialRetryPolicy
[INFO] Running org.apache.hadoop.fs.azurebfs.utils.TestUriUtils
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.713 s - in org.apache.hadoop.fs.azurebfs.services.TestAbfsOutputStream
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.861 s - in org.apache.hadoop.fs.azurebfs.TestAbfsInputStreamStatistics
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.668 s - in org.apache.hadoop.fs.azurebfs.utils.TestUriUtils
[INFO] Running org.apache.hadoop.fs.azurebfs.TestAbfsOutputStreamStatistics
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.598 s - in org.apache.hadoop.fs.azurebfs.services.TestAbfsRenameRetryRecovery
[INFO] Running org.apache.hadoop.fs.azurebfs.diagnostics.TestConfigurationValidators
[INFO] Running org.apache.hadoop.fs.azurebfs.TestAbfsNetworkStatistics
[INFO] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.322 s - in org.apache.hadoop.fs.azurebfs.diagnostics.TestConfigurationValidators
[INFO] Running org.apache.hadoop.fs.azurebfs.TestTracingContext
[INFO] Running org.apache.hadoop.fs.azurebfs.TestAbfsConfigurationFieldsValidation
[INFO] Running org.apache.hadoop.fs.azurebfs.TestAbfsErrorTranslation
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.61 s - in org.apache.hadoop.fs.azurebfs.TestAbfsConfigurationFieldsValidation
[INFO] Running org.apache.hadoop.fs.azurebfs.TestAbfsStatistics
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.534 s - in org.apache.hadoop.fs.azurebfs.TestAbfsErrorTranslation
[INFO] Running org.apache.hadoop.fs.azurebfs.extensions.TestCustomOauthTokenProvider
[INFO] Running org.apache.hadoop.fs.azurebfs.extensions.TestDTManagerLifecycle
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.441 s - in org.apache.hadoop.fs.azurebfs.TestAbfsOutputStreamStatistics
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.849 s - in org.apache.hadoop.fs.azurebfs.extensions.TestCustomOauthTokenProvider
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.786 s - in org.apache.hadoop.fs.azurebfs.TestAbfsNetworkStatistics
[WARNING] Tests run: 3, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 6.956 s - in org.apache.hadoop.fs.azurebfs.TestTracingContext
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.59 s - in org.apache.hadoop.fs.azurebfs.extensions.TestDTManagerLifecycle
[INFO] Running org.apache.hadoop.fs.azurebfs.TestAbfsCrc64
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.061 s - in org.apache.hadoop.fs.azurebfs.TestAbfsCrc64
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.686 s - in org.apache.hadoop.fs.azurebfs.TestAbfsStatistics
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 36.877 s - in org.apache.hadoop.fs.azurebfs.services.TestAbfsClientThrottlingAnalyzer
[INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 55.459 s - in org.apache.hadoop.fs.azurebfs.services.TestAbfsInputStream
[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]   TestAccountConfiguration.testConfigPropNotFound:386->testMissingConfigKey:399 Expected a org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException to be thrown, but got the result: : "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider"
[INFO]
[ERROR] Tests run: 107, Failures: 1, Errors: 0, Skipped: 2
[INFO]
[ERROR] There are test failures.

Please refer to /hadoop/hadoop-tools/hadoop-azure/target/surefire-reports for the individual test results.
Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
[INFO]
[INFO] --- maven-jar-plugin:2.5:jar (default-jar) @ hadoop-azure ---
[INFO]
[INFO] --- maven-jar-plugin:2.5:test-jar (default) @ hadoop-azure ---
[INFO] Building jar: /hadoop/hadoop-tools/hadoop-azure/target/hadoop-azure-3.4.0-SNAPSHOT-tests.jar
[INFO]
[INFO] --- maven-site-plugin:3.11.0:attach-descriptor (attach-descriptor) @ hadoop-azure ---
[INFO] Skipping because packaging 'jar' is not pom.
[INFO]
[INFO] --- maven-failsafe-plugin:3.0.0-M1:integration-test (integration-test-abfs-parallel-classesAndMethods) @ hadoop-azure ---
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit47/3.0.0-M1/surefire-junit47-3.0.0-M1.jar
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit47/3.0.0-M1/surefire-junit47-3.0.0-M1.jar (93 kB at 2.4 MB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit47/3.0.0-M1/surefire-junit47-3.0.0-M1.pom
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit47/3.0.0-M1/surefire-junit47-3.0.0-M1.pom (7.3 kB at 292 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-junit48/3.0.0-M1/common-junit48-3.0.0-M1.pom
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-junit48/3.0.0-M1/common-junit48-3.0.0-M1.pom (2.4 kB at 70 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-grouper/3.0.0-M1/surefire-grouper-3.0.0-M1.pom
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-grouper/3.0.0-M1/surefire-grouper-3.0.0-M1.pom (2.6 kB at 146 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-junit48/3.0.0-M1/common-junit48-3.0.0-M1.jar
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-grouper/3.0.0-M1/surefire-grouper-3.0.0-M1.jar
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-junit48/3.0.0-M1/common-junit48-3.0.0-M1.jar (22 kB at 1.3 MB/s)
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-grouper/3.0.0-M1/surefire-grouper-3.0.0-M1.jar (40 kB at 1.8 MB/s)
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsRestOperationException
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.294 s - in org.apache.hadoop.fs.azurebfs.ITestAbfsRestOperationException
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemRenameUnicode
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.288 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemRenameUnicode
[INFO] Running org.apache.hadoop.fs.azurebfs.services.ITestAbfsOutputStream
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.765 s - in org.apache.hadoop.fs.azurebfs.services.ITestAbfsOutputStream
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCLI
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.222 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCLI
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.588 s - in org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics
[INFO] Running org.apache.hadoop.fs.azurebfs.services.ITestAbfsPositionedRead
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.635 s - in org.apache.hadoop.fs.azurebfs.services.ITestAbfsPositionedRead
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemBackCompat
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.634 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemBackCompat
[INFO] Running org.apache.hadoop.fs.azurebfs.services.ITestAbfsUnbuffer
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.496 s - in org.apache.hadoop.fs.azurebfs.services.ITestAbfsUnbuffer
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCopy
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.503 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCopy
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemAppend
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.634 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemAppend
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemAuthorization
[WARNING] Tests run: 29, Failures: 0, Errors: 0, Skipped: 29, Time elapsed: 2.638 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemAuthorization
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestOauthOverAbfsScheme
[WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.277 s - in org.apache.hadoop.fs.azurebfs.ITestOauthOverAbfsScheme
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemInitAndCreate
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.317 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemInitAndCreate
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsDurationTrackers
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.97 s - in org.apache.hadoop.fs.azurebfs.ITestAbfsDurationTrackers
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemDelete
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.973 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemDelete
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsClient
[WARNING] Tests run: 5, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 19.652 s - in org.apache.hadoop.fs.azurebfs.ITestAbfsClient
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFilesystemAcl
[WARNING] Tests run: 66, Failures: 0, Errors: 0, Skipped: 56, Time elapsed: 11.749 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFilesystemAcl
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestFileSystemProperties
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.013 s - in org.apache.hadoop.fs.azurebfs.ITestFileSystemProperties
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemOauth
[WARNING] Tests run: 2, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 0.099 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemOauth
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemMkDir
[WARNING] Tests run: 5, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.549 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemMkDir
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsOutputStreamStatistics
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.074 s - in org.apache.hadoop.fs.azurebfs.ITestAbfsOutputStreamStatistics
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsIdentityTransformer
[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.745 s - in org.apache.hadoop.fs.azurebfs.ITestAbfsIdentityTransformer
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemFlush
[INFO] Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 23.404 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemFlush
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemE2E
[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.549 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemE2E
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemStoreListStatusWithRange
[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.767 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemStoreListStatusWithRange
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCheckAccess
[ERROR] Tests run: 12, Failures: 1, Errors: 0, Skipped: 10, Time elapsed: 1.521 s <<< FAILURE! - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCheckAccess
[ERROR] testCheckAccessForAccountWithoutNS(org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCheckAccess)  Time elapsed: 1.15 s  <<< FAILURE!
java.lang.AssertionError: Expecting org.apache.hadoop.security.AccessControlException with text "This request is not authorized to perform this operation using this permission.", 403 but got : "void"
        at org.apache.hadoop.test.LambdaTestUtils.intercept(LambdaTestUtils.java:499)
        at org.apache.hadoop.test.LambdaTestUtils.intercept(LambdaTestUtils.java:529)
        at org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCheckAccess.testCheckAccessForAccountWithoutNS(ITestAzureBlobFileSystemCheckAccess.java:181)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
        at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
        at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
        at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
        at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
        at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
        at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.lang.Thread.run(Thread.java:829)

[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsInputStreamStatistics
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.827 s - in org.apache.hadoop.fs.azurebfs.ITestAbfsInputStreamStatistics
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsStatistics
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.649 s - in org.apache.hadoop.fs.azurebfs.ITestAbfsStatistics
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCreate
[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.393 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCreate
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemMainOperation
[INFO] Tests run: 50, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.373 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemMainOperation
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemFinalize
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.362 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemFinalize
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemRandomRead
[WARNING] Tests run: 10, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 3.196 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemRandomRead
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemRename
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 27.622 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemRename
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsHugeFiles
[INFO] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.34 s - in org.apache.hadoop.fs.azurebfs.ITestAbfsHugeFiles
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestWasbAbfsCompatibility
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.802 s - in org.apache.hadoop.fs.azurebfs.ITestWasbAbfsCompatibility
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestClientUrlScheme
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.604 s - in org.apache.hadoop.fs.azurebfs.ITestClientUrlScheme
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestFileSystemRegistration
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.775 s - in org.apache.hadoop.fs.azurebfs.ITestFileSystemRegistration
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestSharedKeyAuth
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.186 s - in org.apache.hadoop.fs.azurebfs.ITestSharedKeyAuth
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestFileSystemInitialization
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.474 s - in org.apache.hadoop.fs.azurebfs.ITestFileSystemInitialization
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestCustomerProvidedKey
[WARNING] Tests run: 28, Failures: 0, Errors: 0, Skipped: 28, Time elapsed: 0.882 s - in org.apache.hadoop.fs.azurebfs.ITestCustomerProvidedKey
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemAttributes
[WARNING] Tests run: 3, Failures: 0, Errors: 0, Skipped: 3, Time elapsed: 0.402 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemAttributes
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemDelegationSAS
[WARNING] Tests run: 14, Failures: 0, Errors: 0, Skipped: 14, Time elapsed: 0.41 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemDelegationSAS
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsMsiTokenProvider
[WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.298 s - in org.apache.hadoop.fs.azurebfs.ITestAbfsMsiTokenProvider
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemFileStatus
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.921 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemFileStatus
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestGetNameSpaceEnabled
[WARNING] Tests run: 8, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 6.37 s - in org.apache.hadoop.fs.azurebfs.ITestGetNameSpaceEnabled
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsListStatusRemoteIterator
[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.053 s - in org.apache.hadoop.fs.azurebfs.ITestAbfsListStatusRemoteIterator
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemPermission
[WARNING] Tests run: 128, Failures: 0, Errors: 0, Skipped: 128, Time elapsed: 9.17 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemPermission
[INFO] Running org.apache.hadoop.fs.azurebfs.services.ITestAbfsInputStreamSmallFileReads
[INFO] Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 48.622 s - in org.apache.hadoop.fs.azurebfs.services.ITestAbfsInputStreamSmallFileReads
[INFO] Running org.apache.hadoop.fs.azurebfs.services.ITestAbfsInputStream
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 48.918 s - in org.apache.hadoop.fs.azurebfs.services.ITestAbfsInputStream
[INFO] Running org.apache.hadoop.fs.azurebfs.services.ITestAbfsInputStreamReadFooter
[INFO] Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 58.717 s - in org.apache.hadoop.fs.azurebfs.services.ITestAbfsInputStreamReadFooter
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemLease
[INFO] Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 75.544 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemLease
[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]   ITestAzureBlobFileSystemCheckAccess.testCheckAccessForAccountWithoutNS:181 Expecting org.apache.hadoop.security.AccessControlException with text "This request is not authorized to perform this operation using this permission.", 403 but got : "void"
[INFO]
[ERROR] Tests run: 567, Failures: 1, Errors: 0, Skipped: 277
[INFO]
[INFO]
[INFO] --- maven-failsafe-plugin:3.0.0-M1:integration-test (integration-test-abfs-parallel-classes) @ hadoop-azure ---
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.fs.azurebfs.commit.ITestAbfsLoadManifestsStage
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestSmallWriteOptimization
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsReadWriteAndSeek
[INFO] Running org.apache.hadoop.fs.azurebfs.services.ITestReadBufferManager
[INFO] Running org.apache.hadoop.fs.azurebfs.commit.ITestAbfsCleanupStage
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemE2EScale
[INFO] Running org.apache.hadoop.fs.azurebfs.commit.ITestAbfsRenameStageFailure
[INFO] Running org.apache.hadoop.fs.azurebfs.commit.ITestAbfsJobThroughManifestCommitter
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.918 s - in org.apache.hadoop.fs.azurebfs.services.ITestReadBufferManager
[INFO] Running org.apache.hadoop.fs.azurebfs.commit.ITestAbfsCommitTaskStage
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.879 s - in org.apache.hadoop.fs.azurebfs.commit.ITestAbfsRenameStageFailure
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.657 s - in org.apache.hadoop.fs.azurebfs.commit.ITestAbfsCleanupStage
[INFO] Running org.apache.hadoop.fs.azurebfs.commit.ITestAbfsTaskManifestFileIO
[INFO] Running org.apache.hadoop.fs.azurebfs.commit.ITestAbfsCreateOutputDirectoriesStage
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 24.991 s - in org.apache.hadoop.fs.azurebfs.commit.ITestAbfsLoadManifestsStage
[INFO] Running org.apache.hadoop.fs.azurebfs.commit.ITestAbfsManifestCommitProtocol
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.569 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemE2EScale
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.159 s - in org.apache.hadoop.fs.azurebfs.commit.ITestAbfsCommitTaskStage
[INFO] Running org.apache.hadoop.fs.azurebfs.commit.ITestAbfsTerasort
[INFO] Running org.apache.hadoop.fs.azurebfs.commit.ITestAbfsManifestStoreOperations
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.197 s - in org.apache.hadoop.fs.azurebfs.commit.ITestAbfsTaskManifestFileIO
[ERROR] Tests run: 19, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 32.672 s <<< FAILURE! - in org.apache.hadoop.fs.azurebfs.commit.ITestAbfsJobThroughManifestCommitter
[ERROR] test_0420_validateJob(org.apache.hadoop.fs.azurebfs.commit.ITestAbfsJobThroughManifestCommitter)  Time elapsed: 0.74 s  <<< ERROR!
org.apache.hadoop.mapreduce.lib.output.committer.manifest.impl.OutputValidationException: `abfs://****@****.dfs.core.windows.net/fork-0002/test/TestJobThroughManifestCommitter/out%20put/dir-03-00/dir-02-00/dir-01-00/part-00-0033': Expected the file renamed from abfs://****@****.dfs.core.windows.net/fork-0002/test/TestJobThroughManifestCommitter/out put/_temporary/202211291549220355_0002/01/tasks/attempt_202211291549220355_0002_m_000000_1/dir-03-00/dir-02-00/dir-01-00/part-00-0033 with etag 0x8DAD22153D4F458 and length 2 but found a file with etag 0x8DAD22159994F7E and length 2
        at org.apache.hadoop.mapreduce.lib.output.committer.manifest.stages.ValidateRenamedFilesStage.validateOneFile(ValidateRenamedFilesStage.java:166)
        at org.apache.hadoop.util.functional.TaskPool$Builder.lambda$runParallel$0(TaskPool.java:410)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)

[INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 33.371 s - in org.apache.hadoop.fs.azurebfs.ITestAbfsReadWriteAndSeek
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsStreamStatistics
[INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractRename
[INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAzureBlobFileSystemBasics
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.86 s - in org.apache.hadoop.fs.azurebfs.commit.ITestAbfsCreateOutputDirectoriesStage
[INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractRootDirectory
[WARNING] Tests run: 3, Failures: 0, Errors: 0, Skipped: 3, Time elapsed: 8.559 s - in org.apache.hadoop.fs.azurebfs.commit.ITestAbfsManifestStoreOperations
[WARNING] Tests run: 48, Failures: 0, Errors: 0, Skipped: 24, Time elapsed: 41.978 s - in org.apache.hadoop.fs.azurebfs.ITestSmallWriteOptimization
[INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractOpen
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.96 s - in org.apache.hadoop.fs.azurebfs.ITestAbfsStreamStatistics
[INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractMkdir
[INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractSetTimes
[WARNING] Tests run: 9, Failures: 0, Errors: 0, Skipped: 9, Time elapsed: 9.624 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractRootDirectory
[INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.207 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractRename
[INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractEtag
[INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsContractUnbuffer
[ERROR] Tests run: 17, Failures: 0, Errors: 7, Skipped: 0, Time elapsed: 25.661 s <<< FAILURE! - in org.apache.hadoop.fs.azurebfs.commit.ITestAbfsManifestCommitProtocol
[ERROR] testCommitterWithDuplicatedCommit(org.apache.hadoop.fs.azurebfs.commit.ITestAbfsManifestCommitProtocol)  Time elapsed: 10.143 s  <<< ERROR!
org.apache.hadoop.mapreduce.lib.output.committer.manifest.impl.OutputValidationException: `abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testCommitterWithDuplicatedCommit/part-m-00000': Expected the file renamed from abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testCommitterWithDuplicatedCommit/_temporary/job_202211291549500881_0004/01/tasks/attempt_202211291549500881_0004_m_000000_0/part-m-00000 with etag 0x8DAD2215D8BA025 and length 40 but found a file with etag 0x8DAD2215F25DA02 and length 40
        at org.apache.hadoop.mapreduce.lib.output.committer.manifest.stages.ValidateRenamedFilesStage.validateOneFile(ValidateRenamedFilesStage.java:166)
        at org.apache.hadoop.util.functional.TaskPool$Builder.lambda$runParallel$0(TaskPool.java:410)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)

[ERROR] testMapFileOutputCommitter(org.apache.hadoop.fs.azurebfs.commit.ITestAbfsManifestCommitProtocol)  Time elapsed: 1.857 s  <<< ERROR!
org.apache.hadoop.mapreduce.lib.output.committer.manifest.impl.OutputValidationException: `abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testMapFileOutputCommitter/part-m-00000/data': Expected the file renamed from abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testMapFileOutputCommitter/_temporary/job_202211291550000055_0004/01/tasks/attempt_202211291550000055_0004_m_000000_0/part-m-00000/data with etag 0x8DAD2215FF58EA2 and length 296 but found a file with etag 0x8DAD221603B8E0D and length 296
        at org.apache.hadoop.mapreduce.lib.output.committer.manifest.stages.ValidateRenamedFilesStage.validateOneFile(ValidateRenamedFilesStage.java:166)
        at org.apache.hadoop.util.functional.TaskPool$Builder.lambda$runParallel$0(TaskPool.java:410)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)

[ERROR] testCommitLifecycle(org.apache.hadoop.fs.azurebfs.commit.ITestAbfsManifestCommitProtocol)  Time elapsed: 1.462 s  <<< ERROR!
org.apache.hadoop.mapreduce.lib.output.committer.manifest.impl.OutputValidationException: `abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testCommitLifecycle/part-m-00000': Expected the file renamed from abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testCommitLifecycle/_temporary/job_202211291550020019_0004/01/tasks/attempt_202211291550020019_0004_m_000000_0/part-m-00000 with etag 0x8DAD22160F41493 and length 40 but found a file with etag 0x8DAD22161466E35 and length 40
        at org.apache.hadoop.mapreduce.lib.output.committer.manifest.stages.ValidateRenamedFilesStage.validateOneFile(ValidateRenamedFilesStage.java:166)
        at org.apache.hadoop.util.functional.TaskPool$Builder.lambda$runParallel$0(TaskPool.java:410)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)

[ERROR] testConcurrentCommitTaskWithSubDir(org.apache.hadoop.fs.azurebfs.commit.ITestAbfsManifestCommitProtocol)  Time elapsed: 1.262 s  <<< ERROR!
org.apache.hadoop.mapreduce.lib.output.committer.manifest.impl.OutputValidationException: `abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testConcurrentCommitTaskWithSubDir/SUB_DIR/part-m-00000': Expected the file renamed from abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testConcurrentCommitTaskWithSubDir/_temporary/job_202211291550030237_0004/00/tasks/attempt_202211291550030237_0004_m_000000_1/SUB_DIR/part-m-00000 with etag 0x8DAD22161DB6C66 and length 40 but found a file with etag 0x8DAD22162100941 and length 40
        at org.apache.hadoop.mapreduce.lib.output.committer.manifest.stages.ValidateRenamedFilesStage.validateOneFile(ValidateRenamedFilesStage.java:166)
        at org.apache.hadoop.util.functional.TaskPool$Builder.lambda$runParallel$0(TaskPool.java:410)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)

[ERROR] testParallelJobsToAdjacentPaths(org.apache.hadoop.fs.azurebfs.commit.ITestAbfsManifestCommitProtocol)  Time elapsed: 1.401 s  <<< ERROR!
org.apache.hadoop.mapreduce.lib.output.committer.manifest.impl.OutputValidationException: `abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testParallelJobsToAdjacentPaths/part-m-00000': Expected the file renamed from abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testParallelJobsToAdjacentPaths/_temporary/job_202211291550080667_0004/01/tasks/attempt_202211291550080667_0004_m_000000_0/part-m-00000 with etag 0x8DAD221646E60AB and length 40 but found a file with etag 0x8DAD22164B4AE25 and length 40
        at org.apache.hadoop.mapreduce.lib.output.committer.manifest.stages.ValidateRenamedFilesStage.validateOneFile(ValidateRenamedFilesStage.java:166)
        at org.apache.hadoop.util.functional.TaskPool$Builder.lambda$runParallel$0(TaskPool.java:410)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)

[ERROR] testOutputFormatIntegration(org.apache.hadoop.fs.azurebfs.commit.ITestAbfsManifestCommitProtocol)  Time elapsed: 0.925 s  <<< ERROR!
org.apache.hadoop.mapreduce.lib.output.committer.manifest.impl.OutputValidationException: `abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testOutputFormatIntegration/part-m-00000': Expected the file renamed from abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testOutputFormatIntegration/_temporary/job_202211291550110055_0004/01/tasks/attempt_202211291550110055_0004_m_000000_0/part-m-00000 with etag 0x8DAD221662398A8 and length 4 but found a file with etag 0x8DAD221664D1399 and length 4
        at org.apache.hadoop.mapreduce.lib.output.committer.manifest.stages.ValidateRenamedFilesStage.validateOneFile(ValidateRenamedFilesStage.java:166)
        at org.apache.hadoop.util.functional.TaskPool$Builder.lambda$runParallel$0(TaskPool.java:410)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)

[ERROR] testTwoTaskAttemptsCommit(org.apache.hadoop.fs.azurebfs.commit.ITestAbfsManifestCommitProtocol)  Time elapsed: 1.291 s  <<< ERROR!
org.apache.hadoop.mapreduce.lib.output.committer.manifest.impl.OutputValidationException: `abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testTwoTaskAttemptsCommit/attempt2-m-00000': Expected the file renamed from abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testTwoTaskAttemptsCommit/_temporary/job_202211291550130427_0004/01/tasks/attempt_202211291550130427_0004_m_000000_1/attempt2-m-00000 with etag 0x8DAD221676CB12D and length 40 but found a file with etag 0x8DAD22167ACBE08 and length 40
        at org.apache.hadoop.mapreduce.lib.output.committer.manifest.stages.ValidateRenamedFilesStage.validateOneFile(ValidateRenamedFilesStage.java:166)
        at org.apache.hadoop.util.functional.TaskPool$Builder.lambda$runParallel$0(TaskPool.java:410)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)

[WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 7.58 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractSetTimes
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.177 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractMkdir
[INFO] Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.624 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractOpen
[INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractDistCp
[INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractAppend
[INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractConcat
[INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractDelete
[INFO] Tests run: 45, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 24.35 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAzureBlobFileSystemBasics
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.322 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsContractUnbuffer
[WARNING] Tests run: 4, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 11.125 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractEtag
[INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractCreate
[WARNING] Tests run: 5, Failures: 0, Errors: 0, Skipped: 5, Time elapsed: 6.888 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractConcat
[INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractGetFileStatus
[INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractSeek
[WARNING] Tests run: 8, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 9.429 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractAppend
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemListStatus
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.189 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractDelete
[INFO] Running org.apache.hadoop.fs.azurebfs.extensions.ITestAbfsDelegationTokens
[INFO] Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.352 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractCreate
[INFO] Tests run: 20, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.206 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractSeek
[INFO] Tests run: 20, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.014 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractGetFileStatus
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.823 s - in org.apache.hadoop.fs.azurebfs.extensions.ITestAbfsDelegationTokens
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.969 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemListStatus
[INFO] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 49.734 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractDistCp
[ERROR] Tests run: 7, Failures: 1, Errors: 0, Skipped: 2, Time elapsed: 100.527 s <<< FAILURE! - in org.apache.hadoop.fs.azurebfs.commit.ITestAbfsTerasort
[ERROR] test_110_teragen(org.apache.hadoop.fs.azurebfs.commit.ITestAbfsTerasort)  Time elapsed: 25.881 s  <<< FAILURE!
java.lang.AssertionError: teragen(1000, abfs://****@****.dfs.core.windows.net/ITestAbfsTerasort/sortin) failed expected:<0> but was:<1>
        at org.junit.Assert.fail(Assert.java:89)
        at org.junit.Assert.failNotEquals(Assert.java:835)
        at org.junit.Assert.assertEquals(Assert.java:647)
        at org.apache.hadoop.fs.azurebfs.commit.ITestAbfsTerasort.executeStage(ITestAbfsTerasort.java:211)
        at org.apache.hadoop.fs.azurebfs.commit.ITestAbfsTerasort.test_110_teragen(ITestAbfsTerasort.java:244)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
        at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
        at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
        at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
        at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
        at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
        at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
        at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.lang.Thread.run(Thread.java:829)

[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]   ITestAbfsTerasort.test_110_teragen:244->executeStage:211->Assert.assertEquals:647->Assert.failNotEquals:835->Assert.fail:89 teragen(1000, abfs://****@****.dfs.core.windows.net/ITestAbfsTerasort/sortin) failed expected:<0> but was:<1>
[ERROR] Errors:
[ERROR]   ITestAbfsJobThroughManifestCommitter.test_0420_validateJob ? OutputValidation ...
[ERROR]   ITestAbfsManifestCommitProtocol.testCommitLifecycle ? OutputValidation `abfs:/...
[ERROR]   ITestAbfsManifestCommitProtocol.testCommitterWithDuplicatedCommit ? OutputValidation
[ERROR]   ITestAbfsManifestCommitProtocol.testConcurrentCommitTaskWithSubDir ? OutputValidation
[ERROR]   ITestAbfsManifestCommitProtocol.testMapFileOutputCommitter ? OutputValidation ...
[ERROR]   ITestAbfsManifestCommitProtocol.testOutputFormatIntegration ? OutputValidation
[ERROR]   ITestAbfsManifestCommitProtocol.testParallelJobsToAdjacentPaths ? OutputValidation
[ERROR]   ITestAbfsManifestCommitProtocol.testTwoTaskAttemptsCommit ? OutputValidation `...
[INFO]
[ERROR] Tests run: 335, Failures: 1, Errors: 8, Skipped: 46

And here are the commands I run on a new instance to get there:

cd /hadoop/hadoop-maven-plugins
mvn install

cd /hadoop/hadoop-tools/hadoop-azure
cp ./src/test/resources/azure-auth-keys.xml.template ./src/test/resources/azure-auth-keys.xml

az login --identity

ACCOUNT_NAME=****
CONTAINER_NAME=****
RESOURCE_GROUP=****
ACCOUNT_ACCESS_KEY=$(az storage account keys list \
      --resource-group "$RESOURCE_GROUP" \
      --account-name "$ACCOUNT_NAME" \
      --output tsv \
      --query '[].{value:value} | [0]')

cat <<EOT > src/test/resources/azure-auth-keys.xml
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
  <!-- Account names without domain -->
  <property>
    <name>fs.azure.hnsTestAccountName</name>
    <value>$ACCOUNT_NAME</value>
  </property>

  <property>
    <name>fs.azure.nonHnsTestAccountName</name>
    <value>$ACCOUNT_NAME</value>
  </property>
</configuration>
EOT
cp ./src/test/resources/accountSettings/accountName_settings.xml.template ./src/test/resources/accountSettings/${ACCOUNT_NAME}_settings.xml

sed -i -e "s/ACCOUNTNAME/$ACCOUNT_NAME/g" ./src/test/resources/accountSettings/${ACCOUNT_NAME}_settings.xml
sed -i -e "s/CONTAINER_NAME/$CONTAINER_NAME/g" ./src/test/resources/accountSettings/${ACCOUNT_NAME}_settings.xml
sed -i -e "s/IS_NAMESPACE_ENABLED/false/g" ./src/test/resources/accountSettings/${ACCOUNT_NAME}_settings.xml
sed -i -e "s#ACCOUNT_KEY#$ACCOUNT_ACCESS_KEY#g" ./src/test/resources/accountSettings/${ACCOUNT_NAME}_settings.xml

dev-support/testrun-scripts/runtests.sh

Attachments

Issue Links

relates to

HADOOP-18408 [ABFS]: ITestAbfsManifestCommitProtocol fails on nonHNS configuration

Resolved

Activity

People

Assignee:: Unassigned

Reporter:: Carl

Votes:: 0 Vote for this issue

Watchers:: 2 Start watching this issue

Dates

Created:: 29/Nov/22 16:52

Updated:: 29/Nov/22 17:22