Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-40431 Introduce "Arbitrary Stateful Processing" in Structured Streaming with Python
  3. SPARK-40432

Introduce GroupStateImpl and GroupStateTimeout in PySpark

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.4.0
    • 3.4.0
    • Structured Streaming
    • None

    Description

      Introduces GroupStateImpl and GroupStateTimeout in PySpark, and updates Scala codebase to support convenient conversion between PySpark implementation and Scala implementation.

      Attachments

        Issue Links

          is related to

          Sub-task - The sub-task of the issue SPARK-40437 Support string representation of durationMs in GroupState.setTimeoutDuration

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Sub-task - The sub-task of the issue SPARK-40438 Support additionalDuration parameter in GroupState.setTimeoutTimestamp

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open
          links to

          Pull request #37889 [Github] Pull Request #37889 (HeartSaVioR)

          Pull request #37889 [Github] Pull Request #37889 (HeartSaVioR)

          Activity

            People

              kabhwan Jungtaek Lim
              kabhwan Jungtaek Lim
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: