Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-5707

Add a portable Flink streaming synthetic source for testing

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0
    • Component/s: runner-flink
    • Labels:
      None

      Description

      Currently there are no built-in streaming sources for portable pipelines. This makes it hard to test streaming functionality in the Python SDK.

      It would be very useful to add a periodic impulse source that (with some configurable frequency) outputs an empty byte array, which can then be transformed as desired inside the python pipeline. More context in this mailing list discussion.

        Attachments

          Activity

            People

            • Assignee:
              mwylde Micah Wylde
              Reporter:
              mwylde Micah Wylde
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 8h
                8h