Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-186

The streaming WindowedWordCount example is broken on Dataflow

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.1.0-incubating
    • Component/s: examples-java
    • Labels:
      None

      Description

      The streaming version of WindowedWordCount example is broken on Dataflow. Currently, the injector cannot start up and emits the error "java.lang.RuntimeException: DataflowWorkerHarness should not be used for non-streaming pipelines". This in turn means that no messages are generated for the streaming pipeline to process.

      The problem is that DataflowExampleUtils uses a batch workflow to inject Google PubSub messages. This workflow reuses the real options and modifies them to be batch, setting isStreaming to false etc. However, with versioned containers, the option for workerHarnessContainerImage must now also be changed to use the batch image.

        Attachments

          Activity

            People

            • Assignee:
              herohde Henning Rohde
              Reporter:
              herohde Henning Rohde
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: