Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.2.0
    • Fix Version/s: 2.3.0
    • Component/s: Structured Streaming
    • Labels:
      None
    • Target Version/s:

      Description

      Complex state-updating and/or timeout-handling logic in mapGroupsWithState functions may require taking decisions based on the current event-time watermark and/or processing-time. Currently, you can use the sql function `current_timestamp` to get the current processing time, but it needs to passed inserted in every row with a select, and then passed through the encoder, which isnt efficient. Furthermore, there is no way to get the current watermark.

      This JIRA is to expose them through the GroupState API.

        Attachments

          Activity

            People

            • Assignee:
              tdas Tathagata Das
              Reporter:
              tdas Tathagata Das
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: