Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-40431 Introduce "Arbitrary Stateful Processing" in Structured Streaming with Python
  3. SPARK-40432

Introduce GroupStateImpl and GroupStateTimeout in PySpark

Attach filesAttach ScreenshotVotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.4.0
    • 3.4.0
    • Structured Streaming
    • None

    Description

      Introduces GroupStateImpl and GroupStateTimeout in PySpark, and updates Scala codebase to support convenient conversion between PySpark implementation and Scala implementation.

      Attachments

        Issue Links

        is related to

        Sub-task - The sub-task of the issue SPARK-40437 Support string representation of durationMs in GroupState.setTimeoutDuration

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Open

        Sub-task - The sub-task of the issue SPARK-40438 Support additionalDuration parameter in GroupState.setTimeoutTimestamp

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Open
        links to

        Pull request #37889 [Github] Pull Request #37889 (HeartSaVioR)

        Pull request #37889 [Github] Pull Request #37889 (HeartSaVioR)

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            kabhwan Jungtaek Lim
            kabhwan Jungtaek Lim
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment