Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-10996

AssertionError: (10, <class 'int'>) when writing TF Records

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: P1
    • Resolution: Unresolved
    • Affects Version/s: 2.22.0
    • Fix Version/s: None
    • Component/s: sdk-py-core
    • Labels:
      None

      Description

      This code snippet:

       

      def create_tfrecord:
        size = np.array([2.0, 3.0])
        tfexample = tf.train.Example( 
           features=tf.train.Features(
              feature=

      {            'size': tf.train.Feature(float_list=tf.train.FloatList(value=size))    }

      ))
        return tfexample.SerializeToString()

       

      ...

      beam.FlatMap(lambda x: create_tfrecord)

      ...

       

      throws this error:

       

      Traceback (most recent call last): File "apache_beam/runners/common.py", line 961, in apache_beam.runners.common.DoFnRunner.process File "apache_beam/runners/common.py", line 726, in apache_beam.runners.common.PerWindowInvoker.invoke_process File "apache_beam/runners/common.py", line 814, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window File "/opt/conda/lib/python3.7/site-packages/apache_beam/io/iobase.py", line 1061, in process self.writer.write(element) File "/opt/conda/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py", line 420, in write self.sink.write_record(self.temp_handle, value) File "/opt/conda/lib/python3.7/site-packages/apache_beam/io/filebasedsink.py", line 146, in write_record self.write_encoded_record(file_handle, self.coder.encode(value)) File "/opt/conda/lib/python3.7/site-packages/apache_beam/coders/coders.py", line 463, in encode return self.get_impl().encode(value) File "apache_beam/coders/coder_impl.py", line 494, in apache_beam.coders.coder_impl.BytesCoderImpl.encode File "apache_beam/coders/coder_impl.py", line 495, in apache_beam.coders.coder_impl.BytesCoderImpl.encode AssertionError: (10, <class 'int'>)

        Attachments

        1. repro.py
          2 kB
          Valliappa Lakshmanan

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              lak@vlakshman.com Valliappa Lakshmanan
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: