[BEAM-14482] Mismatch in event publish_time

• Type: Bug
• Status: Triage Needed
• Priority: P2
• Resolution: Unresolved
• Affects Version/s: 2.36.0
• Component/s: beam-community
• Labels: python
• Assignee: Unassigned
• Reporter: Daljeet Singh
• Created: 17/May/22 19:53
• Updated: 05/Jun/22 00:30

Description

below is the example code, where we are trying to get the publish_time of the pubsub message in our DoFn(). It seems the type has changed in the version of apache beam starting 2.36.0. However, I was not able to find any release notes that talk about this change. Any reference will be helpful.

class ProtoToDictDoFn(beam.DoFn):
def process(self, element, publish_time=beam.DoFn.TimestampParam):
    """
    element is type PubsubMessage(), 
    """
    print('-------------')
    print(type(element.publish_time))
    print(element.publish_time)
    print('-------------')


Output:
— for version 2.35.0 —
<class 'google.protobuf.timestamp_pb2.Timestamp'>
seconds: 1652814206
nanos: 417000000

— for version >= 2.36.0 —
<class 'proto.datetime_helpers.DatetimeWithNanoseconds'>
2022-05-17 19:02:06.314000+00:00 

Which seems to be an issue. As per the google pubsub documentation publish time should be type "google.protobuf.timestamp_pb2.Timestamp"

Any clue, what has changed or caused this issue?