Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-34285 [Umbrella] Test Flink Release 1.19
  3. FLINK-34500

Release Testing: Verify FLINK-33261 Support Setting Parallelism for Table/SQL Sources

Agile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersConvert to IssueLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      This issue aims to verify FLIP-367.

      Volunteers can verify it by following the doc changes. Since currently only the pre-defined DataGen connector and user-defined connector supports setting source parallelism, volunteers can verify it through DataGen Connector.

      The basic steps include:
      1. Start a Flink cluster and submit a Flink SQL Job to the cluster.
      2. In this Flink Job, use the DataGen SQL Connector to generate data.
      3. Specify the parameter scan.parallelism in DataGen connector options as user-defined parallelism instead of default parallelism.
      4. Observe whether the parallelism of the source has changed on the job graph of the Flink Application UI, and whether the shuffle mode is correct.

      If everything is normal, you will see that the parallelism of the source operator is indeed different from that of downstream, and the shuffle mode is rebalanced by default.

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            yunta Yun Tang
            sudewei.sdw SuDewei
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment