Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-9586

Make collection sources parallelisable

    XMLWordPrintableJSON

    Details

    • Type: Wish
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.5.0
    • Fix Version/s: None
    • Component/s: API / DataSet

      Description

      The note in https://ci.apache.org/projects/flink/flink-docs-release-1.5/dev/datastream_api.html#collection-data-sources

      states that Collecitons are mainly there for testing and do not support parallelism. I believe this to be an unnecessary assumption - I'm sure there are plenty of use cases that already have the data they need to distribute ready at hand. It seems strange that a fixed collection of inputs cannot be parallelised by Flink, which would require users to write their Collections into a text file and re-read them just to get parallelisation.

       

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              sinadoom Sina Madani
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: