Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-9014

Update CachingShuffleBatchReader to record weights by size in bytes

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.19.0
    • Component/s: runner-dataflow
    • Labels:
      None

      Description

      Currently the CachingShuffleBatchReader caches based upon the number of batches and not the size of those batches. This task is about updating CachingShuffleBatchReader to cache based on the size of those batches.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                tysonjh Tyson Hamilton
                Reporter:
                lcwik Luke Cwik
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 40m
                  40m