Details
-
Improvement
-
Status: In Progress
-
Major
-
Resolution: Unresolved
-
3.1.0
-
None
-
None
Description
TheĀ first iteration of continuous processing in Spark 2.3 does not work with stateful operators.
Attachments
Issue Links
1.
|
refactor continuous write exec to its own class | Resolved | Jose Torres | |
2.
|
remove restarting iterators hack | Resolved | Jose Torres | |
3.
|
support single partition aggregates | Resolved | Jose Torres | |
4.
|
add flag to remove whitelist of continuous processing operators | Resolved | Unassigned | |
5.
|
create the bottom-of-task RDD with row buffer | Resolved | Jose Torres | |
6.
|
continuous shuffle dependency | Resolved | Unassigned | |
7.
|
create the top-of-task RDD sending rows to the remote buffer | Resolved | Jose Torres | |
8.
|
continuous replacement for ShuffleExchangeExec | Resolved | Unassigned | |
9.
|
Scheduler changes for continuous processing shuffle support | Resolved | Unassigned | |
10.
|
implement continuous processing coalesce(1) | Resolved | Jose Torres | |
11.
|
TCP based shuffle | Resolved | Unassigned |