[HADOOP-16936] Failure in Abfs tests

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 3.2.1
• Component/s: fs/azure, test
• Assignee: Mehakmeet Singh
• Reporter: Mehakmeet Singh
• Created: 24/Mar/20 13:31
• Updated: 26/Mar/20 07:27
• Resolved: 26/Mar/20 07:27

Description

Failures in Abfs tests in hadoop trunk.

Stack traces(Some of them) :

// code placeholder[ERROR] Failures: 
[ERROR]   ITestAbfsContractUnbuffer>AbstractContractUnbufferTest.testMultipleUnbuffers:100->AbstractContractUnbufferTest.validateFullFileContents:132->AbstractContractUnbufferTest.validateFileContents:139->Assert.assertEquals:645->Assert.failNotEquals:834->Assert.fail:88 failed to read expected number of bytes from stream expected:<1024> but was:<-1>
[ERROR]   ITestAbfsContractUnbuffer>AbstractContractUnbufferTest.testUnbufferAfterRead:53->AbstractContractUnbufferTest.validateFullFileContents:132->AbstractContractUnbufferTest.validateFileContents:139->Assert.assertEquals:645->Assert.failNotEquals:834->Assert.fail:88 failed to read expected number of bytes from stream expected:<1024> but was:<-1>
[ERROR]   ITestAbfsContractUnbuffer>AbstractContractUnbufferTest.testUnbufferBeforeRead:63->AbstractContractUnbufferTest.validateFullFileContents:132->AbstractContractUnbufferTest.validateFileContents:139->Assert.assertEquals:645->Assert.failNotEquals:834->Assert.fail:88 failed to read expected number of bytes from stream expected:<1024> but was:<-1>
[ERROR]   ITestAbfsContractUnbuffer>AbstractContractUnbufferTest.testUnbufferMultipleReads:111->AbstractContractUnbufferTest.validateFileContents:139->Assert.assertEquals:645->Assert.failNotEquals:834->Assert.fail:88 failed to read expected number of bytes from stream expected:<128> but was:<-1>
[ERROR]   ITestAbfsContractUnbuffer.testUnbufferOnClosedFile:67->AbstractContractUnbufferTest.validateFullFileContents:132->AbstractContractUnbufferTest.validateFileContents:139->Assert.assertEquals:645->Assert.failNotEquals:834->Assert.fail:88 failed to read expected number of bytes from stream expected:<1024> but was:<-1>
[ERROR]   ITestAbfsFileSystemContractCreate>AbstractContractCreateTest.testCreateFileOverExistingFileNoOverwrite:94->AbstractContractCreateTest.testCreateFileOverExistingFileNoOverwrite:76->Assert.assertEquals:645->Assert.failNotEquals:834->Assert.fail:88 Wrong file length of file abfs://[e-mail]/fork-4/test/testCreateFileOverExistingFileNoOverwrite-builder status: VersionedFileStatus{VersionedFileStatus{path=abfs://[e-mail]/fork-4/test/testCreateFileOverExistingFileNoOverwrite-builder; isDirectory=false; length=0; replication=1; blocksize=268435456; modification_time=1585055630000; access_time=0; owner=mehakmeet.singh; group=staff; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='"0x8D7CFF531D7B237"'} expected:<256> but was:<0>
[ERROR]   ITestAbfsFileSystemContractCreate>AbstractContractCreateTest.testCreateFileUnderFile:337->AbstractContractCreateTest.expectCreateUnderFileFails:402->AbstractContractCreateTest.createFile:436->Assert.assertEquals:645->Assert.failNotEquals:834->Assert.fail:88 Wrong file length of file abfs://[e-mail]/fork-4/test/testCreateFileUnderFile status: VersionedFileStatus{VersionedFileStatus{path=abfs://[e-mail]/fork-4/test/testCreateFileUnderFile; isDirectory=false; length=0; replication=1; blocksize=268435456; modification_time=1585055613000; access_time=0; owner=mehakmeet.singh; group=staff; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='"0x8D7CFF52773EA8C"'} expected:<256> but was:<0>
[ERROR]   ITestAbfsFileSystemContractCreate>AbstractContractCreateTest.testCreateNewFile:66->AbstractContractCreateTest.testCreateNewFile:59->Assert.assertEquals:645->Assert.failNotEquals:834->Assert.fail:88 Wrong file length of file abfs://[e-mail]/fork-4/test/testCreateNewFile status: VersionedFileStatus{VersionedFileStatus{path=abfs://[e-mail]/fork-4/test/testCreateNewFile; isDirectory=false; length=0; replication=1; blocksize=268435456; modification_time=1585055638000; access_time=0; owner=mehakmeet.singh; group=staff; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='"0x8D7CFF536C8E3ED"'} expected:<256> but was:<0>
[ERROR]   ITestAbfsFileSystemContractCreate>AbstractContractCreateTest.testCreateUnderFileSubdir:354->AbstractContractCreateTest.expectCreateUnderFileFails:402->AbstractContractCreateTest.createFile:436->Assert.assertEquals:645->Assert.failNotEquals:834->Assert.fail:88 Wrong file length of file abfs://[e-mail]/fork-4/test/testCreateUnderFileSubdir status: VersionedFileStatus{VersionedFileStatus{path=abfs://[e-mail]/fork-4/test/testCreateUnderFileSubdir; isDirectory=false; length=0; replication=1; blocksize=268435456; modification_time=1585055640000; access_time=0; owner=mehakmeet.singh; group=staff; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='"0x8D7CFF537FFDC55"'} expected:<256> but was:<0>
[ERROR]   ITestAbfsFileSystemContractCreate>AbstractContractCreateTest.testFileStatusBlocksizeNonEmptyFile:281->Assert.assertEquals:645->Assert.failNotEquals:834->Assert.fail:88 Wrong file length of file abfs://[e-mail]/fork-4/test/testFileStatusBlocksizeNonEmptyFile status: VersionedFileStatus{VersionedFileStatus{path=abfs://[e-mail]/fork-4/test/testFileStatusBlocksizeNonEmptyFile; isDirectory=false; length=0; replication=1; blocksize=268435456; modification_time=1585055632000; access_time=0; owner=mehakmeet.singh; group=staff; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='"0x8D7CFF533034150"'} expected:<256> but was:<0>
[ERROR]   ITestAbfsFileSystemContractCreate>AbstractContractCreateTest.testMkdirUnderFile:366->AbstractContractCreateTest.expectMkdirsUnderFileFails:419->AbstractContractCreateTest.createFile:436->Assert.assertEquals:645->Assert.failNotEquals:834->Assert.fail:88 Wrong file length of file abfs://[e-mail]/fork-4/test/testMkdirUnderFile status: VersionedFileStatus{VersionedFileStatus{path=abfs://[e-mail]/fork-4/test/testMkdirUnderFile; isDirectory=false; length=0; replication=1; blocksize=268435456; modification_time=1585055615000; access_time=0; owner=mehakmeet.singh; group=staff; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='"0x8D7CFF528A44690"'} expected:<256> but was:<0>
[ERROR]   ITestAbfsFileSystemContractCreate>AbstractContractCreateTest.testMkdirUnderFileSubdir:376->AbstractContractCreateTest.expectMkdirsUnderFileFails:419->AbstractContractCreateTest.createFile:436->Assert.assertEquals:645->Assert.failNotEquals:834->Assert.fail:88 Wrong file length of file abfs://[e-mail]/fork-4/test/testMkdirUnderFileSubdir status: VersionedFileStatus{VersionedFileStatus{path=abfs://[e-mail]/fork-4/test/testMkdirUnderFileSubdir; isDirectory=false; length=0; replication=1; blocksize=268435456; modification_time=1585055617000; access_time=0; owner=mehakmeet.singh; group=staff; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='"0x8D7CFF529D2813D"'} expected:<256> but was:<0>
[ERROR]   ITestAbfsFileSystemContractCreate>AbstractContractCreateTest.testOverwriteExistingFile:119->AbstractContractCreateTest.testOverwriteExistingFile:103->Assert.assertEquals:645->Assert.failNotEquals:834->Assert.fail:88 Wrong file length of file abfs://[e-mail]/fork-4/test/testOverwriteExistingFile-builder status: VersionedFileStatus{VersionedFileStatus{path=abfs://[e-mail]/fork-4/test/testOverwriteExistingFile-builder; isDirectory=false; length=0; replication=1; blocksize=268435456; modification_time=1585055636000; access_time=0; owner=mehakmeet.singh; group=staff; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='"0x8D7CFF5356D59CC"'} expected:<256> but was:<0>
[ERROR]   ITestAbfsFileSystemContractDistCp>AbstractContractDistCpTest.largeFilesToRemote:452->AbstractContractDistCpTest.largeFiles:536->Assert.assertEquals:645->Assert.failNotEquals:834->Assert.fail:88 wrong length VersionedFileStatus{VersionedFileStatus{path=abfs://[e-mail]/fork-1/test/ITestAbfsFileSystemContractDistCp/largeFilesToRemote/remote/outputDir/inputDir/file1; isDirectory=false; length=0; replication=1; blocksize=268435456; modification_time=1585055616000; access_time=0; owner=mehakmeet.singh; group=staff; permission=rw-r--r--; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}; version='"0x8D7CFF52928F203"'} expected:<2097152> but was:<0>
[ERROR]   ITestAbfsFileSystemContractDistCp>AbstractContractDistCpTest.testDeepDirectoryStructureFromRemote:458->AbstractContractDistCpTest.distCpDeepDirectoryStructure:499->Assert.assertEquals:645->Assert.failNotEquals:834->Assert.fail:88 wrong length DeprecatedRawLocalFileStatus{path=file:/Users/mehakmeet.singh/hadoopfork/hadoop/hadoop-tools/hadoop-azure/target/test-dir/1/ITestAbfsFileSystemContractDistCp/testDeepDirectoryStructureFromRemote/local/outputDir/inputDir/file1; isDirectory=false; length=0; replication=1; blocksize=33554432; modification_time=1585055683000; access_time=1585055684000; owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false} expected:<100> but was:<0>

Issue Links

• relates to: HADOOP-16910 ABFS Streams to update FileSystem.Statistics counters on IO.