Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-260

WindowMappingFn: Know the getSideInputWindow upper bound to release side input resources

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • P2
    • Resolution: Fixed
    • None
    • 2.0.0
    • beam-model
    • None

    Description

      We currently have no static knowledge about the getSideInputWindow function, and runners are thus forced to hold on to all side input state / elements in case a future element reaches back into an earlier side input element.

      Maybe we need an upper bound on lag from current to result of getSideInputWindow so we can have a progressing gc horizon as we do for GKB window state.

      Attachments

        Issue Links

          is superceded by

          Improvement - An improvement or enhancement to an existing feature or task. BEAM-1931 The DirectRunner should garbage collect PCollectionView contents

          • P3 - Non-urgent bugs, features, and improvements
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. BEAM-1932 SideInputHandler should garbage collect PCollectionView contents

          • P3 - Non-urgent bugs, features, and improvements
          • Open
          links to

          Web Link GitHub PR #2284

          Activity

            People

              tgroh Thomas Groh
              mshields822 Mark Shields
              Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: