[FLINK-29817] Published metadata for apache-flink in pypi are inconsistent and causes poetry to fail

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.16.0
• Fix Version/s: 1.17.0, 1.16.1
• Component/s: API / Python
• Labels: pull-request-available

Environment

macos 12.6 (M1)

Poetry Version: 1.2.2
Python Version:  3.9.12

• Assignee: Huang Xingbo
• Reporter: André Casimiro
• Created: 31/Oct/22 14:26
• Updated: 18/Nov/22 07:58
• Resolved: 18/Nov/22 07:58

Description

Hi, 

Following the debug steps described in this github thread I got to the conclusion that the metadata of the apache-flink 1.16.0 package is wrong. And because of that I cannot properly manage my dependencies using poetry.

I can successfully install it with pip (runs with no errors), as stated in the docs:

python -m pip install apache-flink 

But when I try to include the dependency in my poetry project I got the following error:

 

❯ poetry add apache-flink@1.16.0Updating dependencies
Resolving dependencies... (2.0s)Because pemja (0.2.6) depends on numpy (1.21.4)
 and apache-flink (1.16.0) depends on numpy (>=1.14.3,<1.20), pemja (0.2.6) is incompatible with apache-flink (1.16.0).
So, because cv-features depends on apache-flink (1.16.0) which depends on pemja (0.2.6), version solving failed. 

 

I've followed the same debug steps as in this github thread and can confirm that apache-flink has exactly the same problem as described in the thread: the wheel package in pypi has correct dependency metadata but the pypi published don't.

 

❯ pkginfo -f requires_dist /Users/andre/Downloads/apache_flink-1.16.0-cp39-cp39-macosx_11_0_arm64.whl
requires_dist: ['py4j (==0.10.9.3)', 'python-dateutil (==2.8.0)', 'apache-beam (==2.38.0)', 'cloudpickle (==2.1.0)', 'avro-python3 (!=1.9.2,<1.10.0,>=1.8.1)', 'pytz (>=2018.3)', 'fastavro (<1.4.8,>=1.1.0)', 'requests (>=2.26.0)', 'protobuf (<3.18)', 'httplib2 (<=0.20.4,>=0.19.0)', 'apache-flink-libraries (<1.16.1,>=1.16.0)', 'numpy (<1.22.0,>=1.21.4)', 'pandas (<1.4.0,>=1.3.0)', 'pyarrow (<9.0.0,>=5.0.0)', 'pemja (==0.2.6) ; python_full_version >= "3.7" and platform_system != "Windows"'] 

but the pipy json metadata is wrong:

 

 

❯ curl -sL https://pypi.org/pypi/apache-flink/json | jq '.info.requires_dist'[
  "py4j (==0.10.9.3)",
  "python-dateutil (==2.8.0)",
  "apache-beam (==2.38.0)",
  "cloudpickle (==2.1.0)",
  "avro-python3 (!=1.9.2,<1.10.0,>=1.8.1)",
  "pytz (>=2018.3)",
  "fastavro (<1.4.8,>=1.1.0)",
  "requests (>=2.26.0)",
  "protobuf (<3.18)",
  "httplib2 (<=0.20.4,>=0.19.0)",
  "apache-flink-libraries (<1.16.1,>=1.16.0)",
  "numpy (<1.20,>=1.14.3)",
  "pandas (<1.2.0,>=1.0)",
  "pyarrow (<7.0.0,>=0.15.1)",
  "pemja (==0.2.6) ; python_full_version >= \"3.7\" and platform_system != \"Windows\""
]

 

As per this comment, could you please republish the package correcting this metadata information, please? This other comment can help gain more context.

 

Thanks

 

 

Issue Links

• links to: GitHub Pull Request #21342