Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-10305

InMemoryBagUserStateFactory creates a cache token per state cell

    XMLWordPrintableJSON

Details

    Description

      When the state cache is enabled in the Python SDK, the batch mode of the Flink Runner currently only allows a single user state cell because a new cache token is generated for each state cell; the caching code in the Python SDK Harness only supports one cache token per user state handler.

      Theoretically multiple cache tokens would work but would just be adding to the payload. We should make sure to just send a single cache token in batch mode (which is already the case in streaming)

      Attachments

        Issue Links

          Activity

            People

              mxm Maximilian Michels
              mxm Maximilian Michels
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 3h 10m
                  3h 10m