Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-11384

Improve StreamingDataflow worker to avoid escaping the processing key and including with generated json logs

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: P2
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.27.0
    • Component/s: runner-dataflow
    • Labels:
      None

      Description

      As keys can be large escaping the key could be expensive and including it in all the json of logs made during processing is possibly expensive.
      Additionally the sharding key is more useful to correlate with other logging and status pages within dataflow.

        Attachments

          Activity

            People

            • Assignee:
              scwhittle Sam Whittle
              Reporter:
              scwhittle Sam Whittle

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 0.5h
                0.5h

                  Issue deployment