[FLINK-15847] Include flink-ml-api and flink-ml-lib in opt

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Resolved
• Fix Version/s: 1.11.0
• Component/s: Library / Machine Learning
• Labels: pull-request-available
• Assignee: Hequn Cheng
• Reporter: Hequn Cheng
• Created: 03/Feb/20 01:35
• Updated: 27/Feb/20 13:17
• Resolved: 27/Feb/20 13:17

Description

FLIP-39 rebuilds Flink ML pipeline on top of TableAPI which moves Flink ML a step further. Base on it, users can develop their ML jobs and more and more machine learning platforms are providing ML services.

However, the problem now is the jars of flink-ml-api and flink-ml-lib are only exist on maven repo. Whenever users want to submit ML jobs, they can only depend on the ml modules and package a fat jar. This would be inconvenient especially for the machine learning platforms on which nearly all jobs depend on Flink ML modules and have to package a fat jar.

Given this, it would be better to include jars of flink-ml-api and flink-ml-lib in the `opt` folder, so that users can directly use the jars with the binary release. For example, users can move the jars into the `lib` folder or use -j to upload the jars.

Putting the jars in the `opt` folder instead of the `lib` folder is because currently, the ml jars are still optional for the Flink project by default.

Issue Links

• links to: GitHub Pull Request #10995