Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-14341

Flink-python builds with failure: no such option: --prefix

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.10.0
    • Component/s: API / Python
    • Environment:

      Command: mvn clean install -DskipTests

      pip: 7.1.2

      python: 2.7.9

      Description

      [INFO] --- protoc-jar-maven-plugin:3.7.1:run (default) @ flink-python_2.11 ---
      Downloading from nexus: http://nexus.d.xiaomi.net/nexus/content/groups/public/com/github/os72/protoc-jar/3.7.1/protoc-jar-3.7.1.jar
      Downloaded from nexus: http://nexus.d.xiaomi.net/nexus/content/groups/public/com/github/os72/protoc-jar/3.7.1/protoc-jar-3.7.1.jar (10 MB at 29 MB/s)
      [INFO] Protoc version: 3.7.1
      protoc-jar: protoc version: 3.7.1, detected platform: linux-x86_64 (linux/amd64)
      protoc-jar: embedded: bin/3.7.1/protoc-3.7.1-linux-x86_64.exe
      protoc-jar: executing: [/tmp/protocjar8378491914719706170/bin/protoc.exe, --version]
      libprotoc 3.7.1
      [INFO] Protoc command: /tmp/protocjar8378491914719706170/bin/protoc.exe
      [INFO] Input directories:
      [INFO]     /home/liupengcheng/work/git/xiaomi/flink/flink-python/pyflink/proto
      [INFO] Output targets:
      [INFO]     java: /home/liupengcheng/work/git/xiaomi/flink/flink-python/target/generated-sources (add: main, clean: false, plugin: null, outputOptions: null)
      [INFO] /home/liupengcheng/work/git/xiaomi/flink/flink-python/target/generated-sources does not exist. Creating...
      [INFO]     Processing (java): flink-fn-execution.proto
      protoc-jar: executing: [/tmp/protocjar8378491914719706170/bin/protoc.exe, -I/home/liupengcheng/work/git/xiaomi/flink/flink-python/pyflink/proto, --java_out=/home/liupengcheng/work/git/xiaomi/flink/flink-python/target/generated-sources, /home/liupengcheng/work/git/xiaomi/flink/flink-python/pyflink/proto/flink-fn-execution.proto]
      [INFO] Adding generated sources (java): /home/liupengcheng/work/git/xiaomi/flink/flink-python/target/generated-sources
      [INFO] 
      [INFO] --- exec-maven-plugin:1.5.0:exec (Protos Generation) @ flink-python_2.11 ---
      /home/liupengcheng/work/git/xiaomi/flink/flink-python/pyflink/gen_protos.py:49: UserWarning: Installing grpcio-tools is recommended for development.
        warnings.warn('Installing grpcio-tools is recommended for development.')
      WARNING:root:Installing grpcio-tools into /home/liupengcheng/work/git/xiaomi/flink/flink-python/pyflink/../.eggs/grpcio-wheelsUsage:   
        /usr/local/bin/python -m pip install [options] <requirement specifier> [package-index-options] ...
        /usr/local/bin/python -m pip install [options] -r <requirements file> [package-index-options] ...
        /usr/local/bin/python -m pip install [options] [-e] <vcs project url> ...
        /usr/local/bin/python -m pip install [options] [-e] <local project path> ...
        /usr/local/bin/python -m pip install [options] <archive url/path> ...no such option: --prefix
      Process Process-1:
      Traceback (most recent call last):
        File "/usr/local/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
          self.run()
        File "/usr/local/lib/python2.7/multiprocessing/process.py", line 114, in run
          self._target(*self._args, **self._kwargs)
        File "/home/liupengcheng/work/git/xiaomi/flink/flink-python/pyflink/gen_protos.py", line 126, in _install_grpcio_tools_and_generate_proto_files
          '--upgrade', GRPC_TOOLS, "-I"])
        File "/usr/local/lib/python2.7/subprocess.py", line 540, in check_call
          raise CalledProcessError(retcode, cmd)
      CalledProcessError: Command '['/usr/local/bin/python', '-m', 'pip', 'install', '--prefix', '/home/liupengcheng/work/git/xiaomi/flink/flink-python/pyflink/../.eggs/grpcio-wheels', '--build', '/home/liupengcheng/work/git/xiaomi/flink/flink-python/pyflink/../.eggs/grpcio-wheels-build', '--upgrade', 'grpcio-tools>=1.3.5,<=1.14.2', '-I']' returned non-zero exit status 2
      Traceback (most recent call last):
        File "/home/liupengcheng/work/git/xiaomi/flink/flink-python/pyflink/gen_protos.py", line 146, in <module>
          generate_proto_files(force=True)
        File "/home/liupengcheng/work/git/xiaomi/flink/flink-python/pyflink/gen_protos.py", line 91, in generate_proto_files
          raise ValueError("Proto generation failed (see log for details).")
      ValueError: Proto generation failed (see log for details).
      [ERROR] Command execution failed.
      
      

      I find the root cause from the docs: https://pip.pypa.io/en/stable/news/ , I think it's because the `–prefix` option is only supported since pip v8.0.

      In flink build docs, there are no description about the pip version, so I think here we can use the `--install-option="–prefix=xxx"` for better compatibility.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                liupengcheng liupengcheng
                Reporter:
                liupengcheng liupengcheng
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m