[BEAM-7341] Portable Spark: testGlobalCombineWithDefaultsAndTriggers fails

• Type: Bug
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Fix Version/s: 2.14.0
• Component/s: runner-spark
• Labels: portability-spark
• Assignee: Kyle Weaver
• Reporter: Kyle Weaver
• Created: 16/May/19 22:12
• Updated: 16/May/20 14:04
• Resolved: 30/May/19 22:04

Description

PaneInfo for CombineTest.testGlobalCombineWithDefaultsAndTriggers [1] output is incorrect.

isLast: expected true, is false
timing: expected UNKNOWN, is EARLY

No idea yet why this is happening, but commenting out the special GBK translation for non-merging windows [2] seems to fix it.

[1] https://github.com/apache/beam/blob/8403313ea7d63e49974629136c615e379ea874ce/sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/CombineTest.java#L1219-L1242

[2] https://github.com/apache/beam/blob/e98a3a69295afbfc6984fe92c52125929daf6088/runners/spark/src/main/java/org/apache/beam/runners/spark/translation/SparkBatchPortablePipelineTranslator.java#L165-L170

Issue Links

• is duplicated by: BEAM-7177 Spark portable runner fails testGlobalCombineWithDefaultsAndTriggers
• is related to: BEAM-5392 GroupByKey on Spark: All values for a single key need to fit in-memory at once
• links to: GitHub Pull Request #8618