Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-9049

MemoryMonitor thrashing detection is too aggressive for batch workers

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: P2
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: runner-dataflow
    • Labels:

      Description

      In the streaming dataflow worker we've implemented push-back so that we will reduce parallelism when there is memory pressure. Since we cannot do this on the batch worker, it doesn't make sense for our thrashing detection to be so aggressive. We should increase the thresholds used for thrashing detection when running on Batch workloads.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                bhulette Brian Hulette
                Reporter:
                bhulette Brian Hulette
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 40m
                  1h 40m