Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-11049

Unable to execute partial DAG

    XMLWordPrintableJSON

    Details

      Description

      val benv = ExecutionEnvironment.getExecutionEnvironment
      val btEnv = TableEnvironment.getTableEnvironment(benv)
      val data = benv.fromElements("hello world", "hello flink", "hello hadoop")
      data.writeAsText("/Users/jzhang/a.txt", FileSystem.WriteMode.OVERWRITE);
      val table = data.flatMap(line=>line.split("\\s")).
        map(w => (w, 1)).
        toTable(btEnv, 'word, 'number)
      btEnv.registerTable("wc", table)
      btEnv.sqlQuery("select word, count(1) from wc group by word").
        toDataSet[Row].print()
      

      In the above example, the last statement will trigger 2 job execution (writeAsText and print), but what user expect is the print job. The root cause is that currently, flink unable to submit partial dag.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                zjffdu Jeff Zhang
                Reporter:
                zjffdu Jeff Zhang
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: