Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-10603

Large Source Recording for Interarctive Runner

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: P1
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: runner-py-interactive
    • Labels:
      None

      Description

      This changes the Interactive Runner to create a long-running background caching job that is decoupled from the user pipeline. When a user invokes a collect() or show(), it will read from the cache to compute the requested PCollections. Previously, the user would have to wait for the cache to be fully written to. This allows for the user to start experimenting immediately.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              rohdesam Sam Rohde

              Dates

              • Created:
                Updated:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 44h 20m
                44h 20m

                  Issue deployment