Uploaded image for project: 'Apache Gearpump'
  1. Apache Gearpump
  2. GEARPUMP-342

Don't run DataSourceTask with WindowRunner unless necessary

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • 0.8.4
    • 0.8.5
    • None
    • None

    Description

      Currently, a windowed wordcount pipeline like source.flatMap.window.groupBy.count will actually be executed as source.globalWindow.flatMap.groupBy.window.count and globalWindow.flatMap run by a WindowRunner.

      The globalWindow is unnecessary unless source is followed by an aggregation. Further, the globalWindow requires Watermark.MAX to trigger emits which is not obvious for users.

      Attachments

        Issue Links

          Activity

            People

              mauzhang Manu Zhang
              mauzhang Manu Zhang
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Slack

                  Issue deployment