Details
-
Improvement
-
Status: Resolved
-
P2
-
Resolution: Fixed
-
None
Description
The page https://beam.apache.org/documentation/sdks/python-pipeline-dependencies/ recommends managing Python dependencies via requirements files.
This approach is currently inefficient in light of introduction and adoption of PEP-517 by some packages, see: https://lists.apache.org/thread/trljnxo39c0cmff790yff3h8n5okqt3q and the rest of the thread, and does not mention Custom Containers or SDK prebuilding workflows.
We should revise it and document best practices.
Attachments
Issue Links
- is related to
-
BEAM-10147 Clean /tmp/dataflow-requirements-cache before downloading the requirements from requirements.txt
- Open
-
BEAM-680 Python Dataflow stages stale requirements-cache dependencies
- Resolved
-
BEAM-4032 Support staging binary distributions of dependency packages.
- Triage Needed
- links to