Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-7888

Test Multi Process Direct Runner With Largish Data

    XMLWordPrintableJSON

Details

    • Task
    • Status: Open
    • P3
    • Resolution: Unresolved
    • None
    • None
    • sdk-py-core
    • None

    Description

      Filing this as a tracker.

      We can test multiprocess runner with a largish amount of data to the extend that we can do this on Jenkins. This will serve 2 purposes:

      • Find out issues related to multi processing. It would be easier to find rare issues when running over non-trivial data.
      • Serve as a baseline (if not a benchmark) to understand the limits of the multiprocess runner.

      Attachments

        Issue Links

          is related to

          Improvement - An improvement or enhancement to an existing feature or task. BEAM-3645 Support multi-process execution on the FnApiRunner

          • P2 - Default priority. Will be triaged and planned according to community practices.
          • Triage Needed

          Activity

            People

              Unassigned Unassigned
              altay Ahmet Altay
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated: