Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-16550

HadoopS3* tests fail with NullPointerException exceptions

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

      Description

      Logs: https://travis-ci.org/github/apache/flink/jobs/660975486?utm_medium=notification

      All subsequent builds failed as well. It is likely that this commit / FLINK-16014 introduced the issue, as these tests depend on S3 credentials to be available.

      09:38:48.022 [INFO] -------------------------------------------------------
      09:38:48.025 [INFO]  T E S T S
      09:38:48.026 [INFO] -------------------------------------------------------
      09:38:48.657 [INFO] Running org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase
      09:38:48.669 [INFO] Running org.apache.flink.fs.s3hadoop.HadoopS3FileSystemITCase
      09:38:54.541 [ERROR] Tests run: 3, Failures: 0, Errors: 3, Skipped: 0, Time elapsed: 5.88 s <<< FAILURE! - in org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase
      09:38:54.542 [ERROR] testResumeAfterCommit(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase)  Time elapsed: 3.592 s  <<< ERROR!
      java.lang.Exception: Unexpected exception, expected<java.io.IOException> but was<java.lang.NullPointerException>
      	at org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase.testResumeAfterCommit(HadoopS3RecoverableWriterExceptionITCase.java:162)
      
      09:38:54.542 [ERROR] testResumeWithWrongOffset(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase)  Time elapsed: 0.24 s  <<< ERROR!
      java.lang.Exception: Unexpected exception, expected<java.io.IOException> but was<java.lang.NullPointerException>
      	at org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase.testResumeWithWrongOffset(HadoopS3RecoverableWriterExceptionITCase.java:182)
      
      09:38:54.542 [ERROR] testExceptionWritingAfterCloseForCommit(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase)  Time elapsed: 0.448 s  <<< ERROR!
      java.lang.Exception: Unexpected exception, expected<java.io.IOException> but was<java.lang.NullPointerException>
      	at org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterExceptionITCase.testExceptionWritingAfterCloseForCommit(HadoopS3RecoverableWriterExceptionITCase.java:144)
      
      09:38:55.173 [INFO] Running org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase
      09:38:58.737 [ERROR] Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 10.066 s <<< FAILURE! - in org.apache.flink.fs.s3hadoop.HadoopS3FileSystemITCase
      09:38:58.737 [ERROR] testDirectoryListing(org.apache.flink.fs.s3hadoop.HadoopS3FileSystemITCase)  Time elapsed: 3.448 s  <<< ERROR!
      java.io.FileNotFoundException: No such file or directory: s3://[secure]/temp/tests-f37db36e-c116-4c58-a16b-8ca241baae4b/testdir
      
      09:38:59.447 [INFO] Running org.apache.flink.fs.s3hadoop.HadoopS3FileSystemBehaviorITCase
      09:39:01.791 [ERROR] Tests run: 13, Failures: 0, Errors: 13, Skipped: 0, Time elapsed: 6.611 s <<< FAILURE! - in org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase
      09:39:01.797 [ERROR] testCloseWithNoData(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase)  Time elapsed: 2.394 s  <<< ERROR!
      java.lang.NullPointerException
      	at org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testCloseWithNoData(HadoopS3RecoverableWriterITCase.java:186)
      
      09:39:01.798 [ERROR] testCommitAfterPersist(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase)  Time elapsed: 0.191 s  <<< ERROR!
      java.lang.NullPointerException
      	at org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testCommitAfterPersist(HadoopS3RecoverableWriterITCase.java:208)
      
      09:39:01.799 [ERROR] testRecoverWithEmptyState(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase)  Time elapsed: 0.235 s  <<< ERROR!
      java.lang.NullPointerException
      	at org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testResumeAfterMultiplePersist(HadoopS3RecoverableWriterITCase.java:384)
      	at org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testResumeAfterMultiplePersistWithSmallData(HadoopS3RecoverableWriterITCase.java:352)
      	at org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testRecoverWithEmptyState(HadoopS3RecoverableWriterITCase.java:302)
      
      09:39:01.799 [ERROR] testRecoverFromIntermWithoutAdditionalState(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase)  Time elapsed: 0.181 s  <<< ERROR!
      java.lang.NullPointerException
      	at org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testResumeAfterMultiplePersist(HadoopS3RecoverableWriterITCase.java:384)
      	at org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testResumeAfterMultiplePersistWithSmallData(HadoopS3RecoverableWriterITCase.java:352)
      	at org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testRecoverFromIntermWithoutAdditionalState(HadoopS3RecoverableWriterITCase.java:316)
      
      09:39:01.799 [ERROR] testCallingDeleteObjectTwiceDoesNotThroughException(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase)  Time elapsed: 0.181 s  <<< ERROR!
      java.lang.NullPointerException
      	at org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testCallingDeleteObjectTwiceDoesNotThroughException(HadoopS3RecoverableWriterITCase.java:245)
      
      09:39:01.801 [ERROR] testCommitAfterNormalClose(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase)  Time elapsed: 0.174 s  <<< ERROR!
      java.lang.NullPointerException
      	at org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testCommitAfterNormalClose(HadoopS3RecoverableWriterITCase.java:196)
      
      09:39:01.802 [ERROR] testRecoverWithStateWithMultiPart(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase)  Time elapsed: 0.338 s  <<< ERROR!
      java.lang.NullPointerException
      	at org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testResumeAfterMultiplePersist(HadoopS3RecoverableWriterITCase.java:384)
      	at org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testResumeAfterMultiplePersistWithMultiPartUploads(HadoopS3RecoverableWriterITCase.java:364)
      	at org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testRecoverWithStateWithMultiPart(HadoopS3RecoverableWriterITCase.java:330)
      
      09:39:01.803 [ERROR] testRecoverFromIntermWithoutAdditionalStateWithMultiPart(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase)  Time elapsed: 0.486 s  <<< ERROR!
      java.lang.NullPointerException
      	at org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testResumeAfterMultiplePersist(HadoopS3RecoverableWriterITCase.java:384)
      	at org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testResumeAfterMultiplePersistWithMultiPartUploads(HadoopS3RecoverableWriterITCase.java:364)
      	at org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testRecoverFromIntermWithoutAdditionalStateWithMultiPart(HadoopS3RecoverableWriterITCase.java:337)
      
      09:39:01.810 [ERROR] testRecoverWithState(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase)  Time elapsed: 0.199 s  <<< ERROR!
      java.lang.NullPointerException
      	at org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testResumeAfterMultiplePersist(HadoopS3RecoverableWriterITCase.java:384)
      	at org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testResumeAfterMultiplePersistWithSmallData(HadoopS3RecoverableWriterITCase.java:352)
      	at org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testRecoverWithState(HadoopS3RecoverableWriterITCase.java:309)
      
      09:39:01.810 [ERROR] testCleanupRecoverableState(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase)  Time elapsed: 0.202 s  <<< ERROR!
      java.lang.Exception: Unexpected exception, expected<java.io.FileNotFoundException> but was<java.lang.NullPointerException>
      	at org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testCleanupRecoverableState(HadoopS3RecoverableWriterITCase.java:223)
      
      09:39:01.810 [ERROR] testCommitAfterRecovery(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase)  Time elapsed: 0.26 s  <<< ERROR!
      java.lang.NullPointerException
      	at org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testCommitAfterRecovery(HadoopS3RecoverableWriterITCase.java:270)
      
      09:39:01.810 [ERROR] testRecoverAfterMultiplePersistsState(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase)  Time elapsed: 0.165 s  <<< ERROR!
      java.lang.NullPointerException
      	at org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testResumeAfterMultiplePersist(HadoopS3RecoverableWriterITCase.java:384)
      	at org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testResumeAfterMultiplePersistWithSmallData(HadoopS3RecoverableWriterITCase.java:352)
      	at org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testRecoverAfterMultiplePersistsState(HadoopS3RecoverableWriterITCase.java:323)
      
      09:39:01.810 [ERROR] testRecoverAfterMultiplePersistsStateWithMultiPart(org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase)  Time elapsed: 0.735 s  <<< ERROR!
      java.lang.NullPointerException
      	at org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testResumeAfterMultiplePersist(HadoopS3RecoverableWriterITCase.java:384)
      	at org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testResumeAfterMultiplePersistWithMultiPartUploads(HadoopS3RecoverableWriterITCase.java:364)
      	at org.apache.flink.fs.s3hadoop.HadoopS3RecoverableWriterITCase.testRecoverAfterMultiplePersistsStateWithMultiPart(HadoopS3RecoverableWriterITCase.java:344)
      
      09:39:14.711 [WARNING] Tests run: 8, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 15.262 s - in org.apache.flink.fs.s3hadoop.HadoopS3FileSystemBehaviorITCase
      09:39:15.047 [INFO] 
      09:39:15.047 [INFO] Results:
      09:39:15.047 [INFO] 
      09:39:15.047 [ERROR] Errors: 
      09:39:15.047 [ERROR]   HadoopS3FileSystemITCase>AbstractHadoopFileSystemITTest.testDirectoryListing:127 » FileNotFound
      09:39:15.047 [ERROR]   HadoopS3RecoverableWriterExceptionITCase.testExceptionWritingAfterCloseForCommit » 
      09:39:15.047 [ERROR]   HadoopS3RecoverableWriterExceptionITCase.testResumeAfterCommit »  Unexpected e...
      09:39:15.047 [ERROR]   HadoopS3RecoverableWriterExceptionITCase.testResumeWithWrongOffset »  Unexpect...
      09:39:15.047 [ERROR]   HadoopS3RecoverableWriterITCase.testCallingDeleteObjectTwiceDoesNotThroughException:245 » NullPointer
      09:39:15.047 [ERROR]   HadoopS3RecoverableWriterITCase.testCleanupRecoverableState »  Unexpected exce...
      09:39:15.047 [ERROR]   HadoopS3RecoverableWriterITCase.testCloseWithNoData:186 » NullPointer
      09:39:15.047 [ERROR]   HadoopS3RecoverableWriterITCase.testCommitAfterNormalClose:196 » NullPointer
      09:39:15.047 [ERROR]   HadoopS3RecoverableWriterITCase.testCommitAfterPersist:208 » NullPointer
      09:39:15.047 [ERROR]   HadoopS3RecoverableWriterITCase.testCommitAfterRecovery:270 » NullPointer
      09:39:15.047 [ERROR]   HadoopS3RecoverableWriterITCase.testRecoverAfterMultiplePersistsState:323->testResumeAfterMultiplePersistWithSmallData:352->testResumeAfterMultiplePersist:384 » NullPointer
      09:39:15.047 [ERROR]   HadoopS3RecoverableWriterITCase.testRecoverAfterMultiplePersistsStateWithMultiPart:344->testResumeAfterMultiplePersistWithMultiPartUploads:364->testResumeAfterMultiplePersist:384 » NullPointer
      09:39:15.047 [ERROR]   HadoopS3RecoverableWriterITCase.testRecoverFromIntermWithoutAdditionalState:316->testResumeAfterMultiplePersistWithSmallData:352->testResumeAfterMultiplePersist:384 » NullPointer
      09:39:15.047 [ERROR]   HadoopS3RecoverableWriterITCase.testRecoverFromIntermWithoutAdditionalStateWithMultiPart:337->testResumeAfterMultiplePersistWithMultiPartUploads:364->testResumeAfterMultiplePersist:384 » NullPointer
      09:39:15.047 [ERROR]   HadoopS3RecoverableWriterITCase.testRecoverWithEmptyState:302->testResumeAfterMultiplePersistWithSmallData:352->testResumeAfterMultiplePersist:384 » NullPointer
      09:39:15.047 [ERROR]   HadoopS3RecoverableWriterITCase.testRecoverWithState:309->testResumeAfterMultiplePersistWithSmallData:352->testResumeAfterMultiplePersist:384 » NullPointer
      09:39:15.047 [ERROR]   HadoopS3RecoverableWriterITCase.testRecoverWithStateWithMultiPart:330->testResumeAfterMultiplePersistWithMultiPartUploads:364->testResumeAfterMultiplePersist:384 » NullPointer
      09:39:15.047 [INFO] 
      09:39:15.047 [ERROR] Tests run: 26, Failures: 0, Errors: 17, Skipped: 2
      

        Attachments

        Issue Links

          Activity

            People

            • Assignee:
              AHeise Arvid Heise
              Reporter:
              rmetzger Robert Metzger

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 20m
                20m

                  Issue deployment