Uploaded image for project: 'Zeppelin'
  1. Zeppelin
  2. ZEPPELIN-5809

flink.execution.packages are not added to the classpath in Flink Interpreter

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 0.10.1
    • None
    • flink
    • None

    Description

      I've been trying to use flink.execution.packages with the latest version of Flink Interpreter and it looks like even though Zeppelin is able to download all dependencies they're not added to the classpath. The Flink Interpreter process looks like this:

      /usr/lib/jvm/java-8-openjdk-arm64/bin/java Dfile.encoding=UTF
      8 -Dlog4j.configuration=file:///opt/zeppelin/conf/log4j.properties -Dlog4j.configurationFi
      le=file:///opt/zeppelin/conf/log4j2.properties -Dzeppelin.log.file=/opt/zeppelin/logs/zepp
      elin-interpreter-flink-shared_process--flink-zeppelin-cc5bfc486-86xhr.log -Xmx1024m -cp :/
      opt/zeppelin/local-repo/flink/*:/opt/flink/lib/log4j-slf4j-impl-2.17.1.jar:/opt/flink/lib/
      log4j-core-2.17.1.jar:/opt/flink/lib/log4j-api-2.17.1.jar:/opt/flink/lib/log4j-1.2-api-2.1
      7.1.jar:/opt/flink/lib/flink-table_2.12-1.14.5.jar:/opt/flink/lib/flink-shaded-zookeeper-3
      .4.14.jar:/opt/flink/lib/flink-json-1.14.5.jar:/opt/flink/lib/flink-dist_2.12-1.14.5.jar:/
      opt/flink/lib/flink-csv-1.14.5.jar:::/opt/zeppelin/interpreter/zeppelin-interpreter-shaded
      -0.10.1.jar:/opt/flink/opt/flink-python_2.12-1.14.5.jar:/opt/zeppelin/interpreter/flink/ze
      ppelin-flink-0.10.1-2.12.jar org.apache.zeppelin.interpreter.remote.RemoteInterpreterServe
      r 10.1.0.139 40375 flink-shared_process :

      But the dependencies from flink.execution.packages are downloaded to /opt/zeppelin/.m2, which is not specified above. So I keep getting ClassNotFoundExceptions when specifying dependencies this way. But if I move them to /opt/zeppelin/local-repo/flink/, I no longer have any issues.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            sap1ens Yaroslav Tkachenko

            Dates

              Created:
              Updated:

              Slack

                Issue deployment