Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-5758

Load tests for SyntheticSources in Python

    XMLWordPrintableJSON

    Details

    • Type: Test
    • Status: Open
    • Priority: P3
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: testing
    • Labels:

      Description

      For purpose of load testing the SyntheticSources there should be created tests with metrics which will be sent to BigQuery.

        Attachments

          Issue Links

          1.
          Create ParDo Load Test Sub-task Resolved Kasia Kucharczyk  
          2.
          Create Combine Load Test Sub-task Resolved Kasia Kucharczyk  
          3.
          Create CoGroupByKey Load Test Sub-task Resolved Kasia Kucharczyk  
          4.
          Create GroupByKey Load Test Sub-task Resolved Kasia Kucharczyk  
          5.
          Create SideInput Load test Sub-task Resolved Kasia Kucharczyk

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 1h 40m
          6.
          Publish metrics from load tests to BigQuery database Sub-task Resolved Kasia Kucharczyk

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 2h 50m
          7.
          Create Smoke and GBK Python Load Test Jenkins Job Sub-task Resolved Kasia Kucharczyk

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 53h 10m
          8.
          GroupByKey uses data insertion pipeline in streaming tests Sub-task In Progress Unassigned

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 50m
          9.
          Remove runninig job twice Sub-task Resolved Kasia Kucharczyk

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 50m
          10.
          Create ParDo Python Load Test Jenkins Job Sub-task Resolved Kamil Wasilewski

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 8h 10m
          11.
          Create CoGBK Python Load Test Job Jenkins Sub-task Resolved Kamil Wasilewski

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 1h 10m
          12.
          Create Combine Python Load Test Jenkins Job Sub-task Resolved Kamil Wasilewski

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 7h 20m
          13.
          Create SideInput Python Load Test Jenkins Job Sub-task Open Kamil Wasilewski

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 9h 40m
          14.
          Implement missing pipeline parameters in ParDo Load Test Sub-task Resolved Kamil Wasilewski

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 4h 10m
          15.
          Create ParDo Python Load Test Jenkins Job [Flink] Sub-task Resolved Kamil Wasilewski

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 4h 40m
          16.
          Create CoGBK Python Load Test Jenkins Job [Flink] Sub-task Resolved Kamil Wasilewski

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 0.5h
          17.
          Create Combine Python Load Test Jenkins Job [Flink] Sub-task Resolved Kamil Wasilewski

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 3h 40m
          18.
          Create SideInput Python Load Test Jenkins Job [Flink] Sub-task Open Unassigned  
          19.
          Add the rest of GBK tests in Jenkins job [Flink] Sub-task Resolved Kasia Kucharczyk

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 10h 40m
          20.
          Simpler Flink cluster set up in load tests Sub-task Resolved Kamil Wasilewski

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 15.5h
          21.
          Do not publish any metrics in Load tests on Flink Sub-task Resolved Kamil Wasilewski

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 0.5h
          22.
          Upgrade supported Flink version to 1.9 Sub-task Resolved Kamil Wasilewski

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 5h 40m
          23.
          Turn on metrics publishing in load tests on Flink Sub-task Resolved Kamil Wasilewski

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 2.5h
          24.
          Migrate Python version to 3.7 Sub-task Resolved Kamil Wasilewski

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 15h 40m
          25.
          Add stateful processing to ParDo load test Sub-task Resolved Maximilian Michels

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 16h 10m
          26.
          Bring back the names of the runtime metrics to "runtime" Sub-task Resolved Unassigned  
          27.
          Create a common place for Load Tests configuration Sub-task Open Unassigned

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 5h 50m
          28.
          Stop using nose in load tests Sub-task Resolved Kamil Wasilewski

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 12h
          29.
          Include Dataflow Job ID in console output Sub-task Resolved Kamil Wasilewski  
          30.
          Create Streaming ParDo Python Load Test Jenkins Job Sub-task In Progress Kasia Kucharczyk

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 1h 50m

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                kasiak Kasia Kucharczyk
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 188h
                  188h