[BEAM-10293] Duplicated spark DAG graph by beam

• Type: Bug
• Status: Triage Needed
• Priority: P3
• Resolution: Invalid
• Affects Version/s: 2.8.0
• Fix Version/s: Not applicable
• Component/s: beam-community, beam-model
• Assignee: Aizhamal Nurmamat kyzy
• Reporter: tim
• Created: 22/Jun/20 11:02
• Updated: 01/Sep/24 06:53
• Resolved: 22/Jun/20 11:03

Description

Hi, 
We are using the beam to read the stream from Kafka and using spark as the runner, and I found our spark graph looks very strange.
 
For each batch stream, it will generate 3 stages, one of them of our actual work, that I can understand.
 
Another two is kind of duplicated, you can take a look at the attached graphs:

 
 

I don't see any reason we display two graphs but one of them include another one.
 
Br,
Tim