Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-12639 FLIP-42: Rework Documentation
  3. FLINK-12749

Getting Started - Docker Playgrounds - Flink Cluster Playground

    XMLWordPrintableJSON

Details

    Description

      The planned structure for the new Getting Started Guide is

      • Flink Overview (~ two pages)
      • Project Setup
        • Java
        • Scala
        • Python
      • Quickstarts
        • Example Walkthrough - Table API / SQL
        • Example Walkthrough - DataStream API
      • Docker Playgrounds
        • Flink Cluster Playground
        • Flink Interactive SQL Playground

      In this ticket we add the Flink Cluster Playground, a docker-compose based setup consisting of Apache Kafka and Apache Flink (Flink Session Cluster), including a step-by-step guide for some common commands (job submission, savepoints, etc).

      Some Open Questions:

      • Which Flink images to use? `library/flink` with dynamic properties would be the most maintainable, I think. It would be preferable, if we don't need to host any custom images for this, but can rely on the existing plain Flink images.
      • Which Flink jobs to use? An updated version org.apache.flink.streaming.examples.statemachine.StateMachineExample might be a good option as it can with or without Kafka and contains a data generator writing to Kafka already (see next questions).
      • How to get data into Kafka? Maybe just provide a small bash script/one-liner to produce into Kafka topic or see question above.
      • Which Kafka Images to use? https://hub.docker.com/r/wurstmeister/kafka/ seems to be well-maintained and is openly available.

      Attachments

        Issue Links

          Activity

            People

              knaufk Konstantin Knauf
              knaufk Konstantin Knauf
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 20m
                  1h 20m