Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-12457

beam crash when annotating function with typing.Protocol

Details

    • Bug
    • Status: Triage Needed
    • P3
    • Resolution: Unresolved
    • 2.29.0, 2.31.0
    • None
    • sdk-py-core
    • None

    Description

      When annotating a function with a https://docs.python.org/3/library/typing.html#typing.Protocol:

      class Data(typing.Protocol):
          def f(self) -> int:
              ...
      
      
      def fn(x: Data) -> Data:
        ...
      
      pipeline = (
         ...
         | beam.Map(fn)
         ...
      )
      

      Beam will crash when `pipeline_type_check=True` because isinstance(x, Data) raise an error.
      Indeed, by default, Protocol are not runtime checkable.

      ```

        File "/usr/local/google/_blaze_epot/e52e0fa21f849d3c6badad5aaa8158a8/execroot/google3/blaze-out/k8-fastbuild/bin/third_party/py/tensorflow_datasets/scripts/tools/compute_split_info.runfiles/google3/third_party/py/apache_beam/transforms/ptransform.py", line 935, in type_check_inputs
          hint):
        File "/usr/local/google/_blaze_epot/e52e0fa21f849d3c6badad5aaa8158a8/execroot/google3/blaze-out/k8-fastbuild/bin/third_party/py/tensorflow_datasets/scripts/tools/compute_split_info.runfiles/google3/third_party/py/apache_beam/typehints/typehints.py", line 1172, in is_consistent_with
          return issubclass(sub, base)
        File "/usr/local/google/_blaze_epot/e52e0fa21f849d3c6badad5aaa8158a8/execroot/google3/blaze-out/k8-fastbuild/bin/third_party/py/tensorflow_datasets/scripts/tools/compute_split_info.runfiles/google3/third_party/py/typing_extensions/src_py3/typing_extensions.py", line 1265, in __subclasscheck__
          raise TypeError("Instance and class checks can only be used with"
      TypeError: Instance and class checks can only be used with @runtime protocols
      

      ```

      Even if protocol are not runtime checkable, this is a perfectly valid python annotation, so beam should not raise error. For example, beam could ignore the annotation if isinstance raise error

      Attachments

        Activity

          People

            Unassigned Unassigned
            conchylicultor Etienne POT
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: