Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-4032

Support staging binary distributions of dependency packages.

Details

    • Improvement
    • Status: Resolved
    • P2
    • Resolution: Fixed
    • None
    • 2.38.0
    • sdk-py-core
    • None

    Description

      requirements.txt only supports source-distribution dependencies [1].

      --extra_packages does not officially support wheel files [2].

      It is possible to expand this to support binary distributions as long as we have the knowledge of the target platform.

      We should take into consideration the mechanisms of staging dependencies through portability framework, and perhaps consolidate some of the existing options.

      https://github.com/apache/beam/blob/a79d1b4fc27eb81db0d9a773047820a206f3d238/sdks/python/apache_beam/runners/dataflow/internal/dependency.py#L260

      https://github.com/apache/beam/blob/a79d1b4fc27eb81db0d9a773047820a206f3d238/sdks/python/apache_beam/runners/dataflow/internal/dependency.py#L188

       

      Attachments

        Issue Links

          Activity

            People

              Anand Inguva Anand Inguva
              tvalentyn Valentyn Tymofieiev
              Votes:
              1 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 25h 50m
                  25h 50m

                  Slack

                    Issue deployment