[SPARK-23788] Race condition in StreamingQuerySuite

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.4.0
• Fix Version/s: 2.2.2, 2.3.1, 2.4.0
• Component/s: Structured Streaming
• Assignee: Jose Torres
• Reporter: Jose Torres
• Created: 24/Mar/18 02:13
• Updated: 25/Mar/18 01:29
• Resolved: 25/Mar/18 01:22

Description

The serializability test uses the same MemoryStream instance for 3 different queries. If any of those queries ask it to commit before the others have run, the rest will see empty dataframes. This can fail the test if q3 is affected.

Issue Links

• links to: [Github] Pull Request #20896 (jose-torres)