[FLINK-26628] Use REST program arguments in StatefulFunctionsJob

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Won't Do
• Component/s: Stateful Functions
• Assignee: Unassigned
• Reporter: Fil Karnicki
• Created: 14/Mar/22 08:33
• Updated: 28/Mar/22 16:10
• Resolved: 28/Mar/22 16:10

Description

Currently the Program Arguments passed into the REST api don't get used by StreamingExecutionEnvironment in StatefulFunctionsJob (in this case the checkpointing will not be set)

--execution.checkpointing.interval 1000 --state.backend rocksdb --state.checkpoint-storage filesystem --state.checkpoints.dir file:///tmp/ --statefun.embedded true 

Conversely, Flink CLI params do get used by the StreamingExecutionEnvironment in statefun jobs

flink run -Dexecution.checkpointing.interval=1000 -Dstate.backend=rocksdb -Dstate.checkpoint-storage=filesystem -Dstate.checkpoints.dir=file:///tmp/ -Dstatefun.embedded=true myjar.jar

 

To reproduce,

1. clone and run mvn package on https://github.com/FilKarnicki/statefun-flinkjob/tree/argsNotUsedViaRest
2. run docker-compose up in flinkjob/docker-compose
3. observe checkpointing happening for this job
4. go to http://localhost:8081/#/submit and submit flinkjob-1.0-SNAPSHOT.jar again manually from target with program arguments

 --execution.checkpointing.interval 1000 --state.backend rocksdb --state.checkpoint-storage filesystem --state.checkpoints.dir file:///tmp/ --statefun.embedded true   

        5. observe no checkpointing happening for the second job