Details
-
Umbrella
-
Status: Open
-
Major
-
Resolution: Unresolved
-
3.4.0
-
None
-
None
Description
This is a part of effort for SPARK-39590, python API parity in Structured Streaming.
Most of public APIs are available in both Scala/Java Spark and PySpark, but we have a huge gap on streaming workload in PySpark as we don't have matching API for flatMapGroupsWithState in PySpark.
This ticket is to track the effort.
Attachments
Issue Links
- duplicates
-
SPARK-39590 Python API Parity in Structure Streaming
- Resolved
1.
|
Support additionalDuration parameter in GroupState.setTimeoutTimestamp | Open | Unassigned | |
2.
|
Support string representation of durationMs in GroupState.setTimeoutDuration | Open | Unassigned | |
3.
|
Support applyInPandasWithState in batch query | Open | Unassigned | |
4.
|
Support initial state in applyInPandasWithState | Open | Unassigned | |
5.
|
Improving testability of GroupState in applyInPandasWithState | Open | Unassigned |