Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-9488

Python SDK sending unexpected MonitoringInfo

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.21.0
    • Component/s: sdk-py-core
    • Labels:

      Description

      element_count metrics is supposed to be tied with pcollection ids, but by inspecting what is sent over by python sdk, we see there are monitoringInfo sent wit ptransforms in it.

      [Double checked the job graph, these seem to be redundant. i.e. the corresponding pcollection does have its own MonitoringInfo reported.]

      Likely a bug.

      Proof:

      urn: "beam:metric:element_count:v1"
      type: "beam:metrics:sum_int_64"
      metric {
      counter_data

      { int64_value: 1 }

      }
      labels

      { key: "PTRANSFORM" value: "start/MaybeReshuffle/Reshuffle/RemoveRandomKeys-ptransform-85" }

      labels

      { key: "TAG" value: "None" }

      timestamp

      { seconds: 1583949073 nanos: 842402935 }

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                lcwik Luke Cwik
                Reporter:
                ruoyun Ruoyun Huang
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 3.5h
                  3.5h