Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-2887 Python SDK support for portable pipelines
  3. BEAM-3883

Python SDK stages artifacts when talking to job server

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.6.0
    • Component/s: sdk-py-core
    • Labels:
      None

      Description

      The Python SDK does not currently stage its user-defined functions or dependencies when talking to the job API. Artifacts that need to be staged include the user code itself, any SDK components not included in the container image, and the list of Python packages that must be installed at runtime.

       

      Artifacts that are currently expected can be found in the harness boot code: https://github.com/apache/beam/blob/58e3b06bee7378d2d8db1c8dd534b415864f63e1/sdks/python/container/boot.go#L52.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                angoenka Ankur Goenka
                Reporter:
                bsidhom Ben Sidhom
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 19h 40m
                  19h 40m