[SPARK-21733] ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 2.1.1
• Component/s: DStreams

Environment

Apache Spark2.1.1
CDH5.12.0 Yarn

• Assignee: Unassigned
• Reporter: Jepson
• Created: 15/Aug/17 02:12
• Updated: 18/Feb/19 07:55
• Resolved: 22/Aug/17 05:45

Description

Kafka+Spark streaming ,throw these error:

17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003_piece0 stored as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
17/08/15 09:34:14 INFO broadcast.TorrentBroadcast: Reading broadcast variable 8003 took 11 ms
17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003 stored as values in memory (estimated size 2.9 KB, free 1643.2 MB)
17/08/15 09:34:14 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the same as ending offset skipping kssh 5
17/08/15 09:34:14 INFO executor.Executor: Finished task 7.0 in stage 8003.0 (TID 64178). 1740 bytes result sent to driver
17/08/15 09:34:21 INFO storage.BlockManager: Removing RDD 8002
17/08/15 09:34:21 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 64186
17/08/15 09:34:21 INFO executor.Executor: Running task 7.0 in stage 8004.0 (TID 64186)
17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Started reading broadcast variable 8004
17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004_piece0 stored as bytes in memory (estimated size 1895.0 B, free 1643.2 MB)
17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Reading broadcast variable 8004 took 8 ms
17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004 stored as values in memory (estimated size 2.9 KB, free 1643.2 MB)
17/08/15 09:34:21 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the same as ending offset skipping kssh 5
17/08/15 09:34:21 INFO executor.Executor: Finished task 7.0 in stage 8004.0 (TID 64186). 1740 bytes result sent to driver
h3. 17/08/15 09:34:29 ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
17/08/15 09:34:29 INFO storage.DiskBlockManager: Shutdown hook called
17/08/15 09:34:29 INFO util.ShutdownHookManager: Shutdown hook called