Details
-
Bug
-
Status: Resolved
-
P1
-
Resolution: Fixed
-
None
-
None
Description
Currently, Dataflow metrics processor expects Dataflow internal step names generated for v1beta3 job description in metrics returned by Dataflow service: https://github.com/apache/beam/blob/master/sdks/python/apache_beam/runners/dataflow/dataflow_metrics.py#L97
But with portable job submission, Dataflow uses PTransform ID (in proto pipeline) as the internal step name. Hence metrics processor should be updated to handle this.