Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-11065

Apache Beam Template to ingest from Apache Kafka to Google Pub/Sub

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: P3
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: 2.27.0
    • Component/s: examples-java
    • Labels:

      Description

      In the Beam repository, we have awesome examples on java and python, however, there are not presented any templates.

      The reason for templates here, some users would like to use parametrized flex templates for common use cases. For example, if I want to do a typical pipeline, read from some source, and write to another source, I would happy if Beam provides a finally implemented pipeline for my common case in which I can put only parameters to customize input and output sources.

      I propose to add the new directory under the examples folder and name it "templates". It will be a new Gradle module in the project. As a first template, I propose to implement Kafka to Pubsub template.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                IKozyrev Ilya Kozyrev
                Reporter:
                IKozyrev Ilya Kozyrev
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 6h 50m
                  6h 50m