[SAMZA-1446] Flaky test: TestHdfsSystemProducerTestSuite

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Xinyu Liu
• Reporter: Prateek Maheshwari
• Created: 06/Oct/17 18:12
• Updated: 11/Oct/17 21:44

Description

Failed on my laptop during a check-all.sh run. Looks like the file path (/user/pmahehsw/) is incorrect? Is it hardcoded for linux?

:samza-hdfs_2.10:test
testHdfsSystemProducerTextWrite FAILED
java.io.FileNotFoundException: File /user/pmaheshw/samza-hdfs-test-job-text/2017_10_06-10 does not exist.
at org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:795)
at org.apache.hadoop.hdfs.DistributedFileSystem.access$700(DistributedFileSystem.java:106)
at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:853)
at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:849)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:860)
at org.apache.samza.system.hdfs.TestHdfsSystemProducerTestSuite.testHdfsSystemProducerTextWrite(TestHdfsSystemProducerTestSuite.scala:204)

testHdfsSystemProducerWriteTextBatches FAILED
java.io.FileNotFoundException: File /user/pmaheshw/samza-hdfs-test-batch-job-text/2017_10_06-10 does not exist.
at org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:795)
at org.apache.hadoop.hdfs.DistributedFileSystem.access$700(DistributedFileSystem.java:106)
at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:853)
at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:849)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:860)
at org.apache.samza.system.hdfs.TestHdfsSystemProducerTestSuite.testHdfsSystemProducerWriteTextBatches(TestHdfsSystemProducerTestSuite.scala:242)

testHdfsSystemProducerAvroWrite FAILED
java.io.FileNotFoundException: File /user/pmaheshw/samza-hdfs-test-job-avro/2017_10_06-10 does not exist.
at org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:795)
at org.apache.hadoop.hdfs.DistributedFileSystem.access$700(DistributedFileSystem.java:106)
at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:853)
at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:849)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:860)
at org.apache.samza.system.hdfs.TestHdfsSystemProducerTestSuite.testHdfsSystemProducerAvroWrite(TestHdfsSystemProducerTestSuite.scala:281)

testHdfsSystemProducerWriteAvroBatches FAILED
java.io.FileNotFoundException: File /user/pmaheshw/samza-hdfs-test-batch-job-avro/2017_10_06-10 does not exist.
at org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:795)
at org.apache.hadoop.hdfs.DistributedFileSystem.access$700(DistributedFileSystem.java:106)
at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:853)
at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:849)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:860)
at org.apache.samza.system.hdfs.TestHdfsSystemProducerTestSuite.testHdfsSystemProducerWriteAvroBatches(TestHdfsSystemProducerTestSuite.scala:321)

33 tests completed, 4 failed
:samza-hdfs_2.10:test FAILED

Issue Links

• is depended upon by: SAMZA-1190 Flaky unit tests