Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-15847

Include flink-ml-api and flink-ml-lib in opt

    XMLWordPrintableJSON

    Details

      Description

      FLIP-39 rebuilds Flink ML pipeline on top of TableAPI which moves Flink ML a step further. Base on it, users can develop their ML jobs and more and more machine learning platforms are providing ML services.

      However, the problem now is the jars of flink-ml-api and flink-ml-lib are only exist on maven repo. Whenever users want to submit ML jobs, they can only depend on the ml modules and package a fat jar. This would be inconvenient especially for the machine learning platforms on which nearly all jobs depend on Flink ML modules and have to package a fat jar.

      Given this, it would be better to include jars of flink-ml-api and flink-ml-lib in the `opt` folder, so that users can directly use the jars with the binary release. For example, users can move the jars into the `lib` folder or use -j to upload the jars.

      Putting the jars in the `opt` folder instead of the `lib` folder is because currently, the ml jars are still optional for the Flink project by default.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                hequn8128 Hequn Cheng
                Reporter:
                hequn8128 Hequn Cheng
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m