[BEAM-5788] wordcount_fnapi_it failed on TestDataflowRunner because of JSON string decoding error

• Type: Sub-task
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Fix Version/s: 2.10.0
• Component/s: test-failures
• Assignee: Mark Liu
• Reporter: Mark Liu
• Created: 18/Oct/18 17:01
• Updated: 11/Jun/20 18:59
• Resolved: 04/Jan/19 00:41

Description

Similar to BEAM-5785, wordcount_fnapi_it failed on Python 3 when running with TestDataflowRunner. Got TypeError: the JSON object must be str, not 'bytes'. This error cause infinite retry before job could submitted to service.

More details about my env and test:
Python version: 3.5.3
Test: apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_fnapi_it
Command:

python setup.py nosetests \
  --tests apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_fnapi_it  \
  --nocapture \
  --nologcapture \
  --test-pipeline-options=" \                             
        --runner=TestDataflowRunner \
        --project=<my_project> \
        --staging_location=<my_staging> \
        --temp_location=<my_temp> \
        --output=<my_output> \                                                                      
        --sdk_location=.../beam/sdks/python/dist/apache-beam-2.9.0.dev0.tar.gz \
        --num_workers=1"

Stacktrace:

WARNING:root:Retry with exponential backoff: waiting for 7.661876827680761 seconds before retrying exists because we caught exception: TypeError: the JSON object must be str, not 'bytes'
 Traceback for above exception (most recent call last):
  File ".../beam/sdks/python/apache_beam/utils/retry.py", line 184, in wrapper
    return fun(*args, **kwargs)
  File ".../beam/sdks/python/apache_beam/io/gcp/gcsio.py", line 375, in exists
    self.client.objects.Get(request)  # metadata
  File ".../beam/sdks/python/apache_beam/io/gcp/internal/clients/storage/storage_v1_client.py", line 955, in Get
    download=download)
  File ".../tmp/virtualenvs/py3-env/lib/python3.5/site-packages/apitools/base/py/base_api.py", line 722, in _RunMethod
    return self.ProcessHttpResponse(method_config, http_response, request)
  File ".../tmp/virtualenvs/py3-env/lib/python3.5/site-packages/apitools/base/py/base_api.py", line 728, in ProcessHttpResponse
    self.__ProcessHttpResponse(method_config, http_response, request))
  File ".../tmp/virtualenvs/py3-env/lib/python3.5/site-packages/apitools/base/py/base_api.py", line 611, in __ProcessHttpResponse
    response_type, http_response.content)
  File ".../tmp/virtualenvs/py3-env/lib/python3.5/site-packages/apitools/base/py/base_api.py", line 442, in DeserializeMessage
    message = encoding.JsonToMessage(response_type, data)
  File ".../tmp/virtualenvs/py3-env/lib/python3.5/site-packages/apitools/base/py/encoding.py", line 104, in JsonToMessage
    return _ProtoJsonApiTools.Get().decode_message(message_type, message)
  File ".../tmp/virtualenvs/py3-env/lib/python3.5/site-packages/apitools/base/py/encoding.py", line 290, in decode_message
    message_type, result)
  File ".../tmp/virtualenvs/py3-env/lib/python3.5/site-packages/apitools/base/protorpclite/protojson.py", line 210, in decode_message
    dictionary = json.loads(encoded_message)
  File "/usr/lib/python3.5/json/__init__.py", line 312, in loads
    s.__class__.__name__))

Issue Links

• relates to: BEAM-5946 Upgrade google-apitools to 0.5.24
• links to: GitHub Pull Request #7010
• links to: GitHub Pull Request #7017
• links to: GitHub Pull Request #7026