[THRIFT-5488] SystemError when using fast binary or compact protocol in python 3.10 - ASF JIRA

Attach files

Attach Screenshot

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Bug
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: 0.15.0
Fix Version/s: 0.16.0
Component/s: Python - Library
Labels:
None

Language:
- Python

Description

Up till python 3.9 a DeprecationWarning was raised when '#' formats were used without defining PY_SSIZE_T_CLEAN:

DeprecationWarning: PY_SSIZE_T_CLEAN will be required for '#' formats
  iprot._fast_decode(self, iprot, [self.__class__, self.thrift_spec])

Starting with python 3.10 this now becomes a runtime SystemError. Details of change: Python bug tracker 40943

This specifically gets exercised in the c python extension module in ProtocolBase<Impl>::readBytes (src/protocol.tcc) when falling back to the building function.

An easy way to reproduce the error is to pass an empty byte array to the TSerializer.deserialize method, for example:

  buf = TTransport.TMemoryBuffer()
  transport = TTransport.TBufferedTransportFactory().getTransport(buf)
  factory = TBinaryProtocolAcceleratedFactory(transport)
  TSerialization.deserialize(Message(), b'', factory) #  Message is a thrift struct. This raises a SystemError in python 3.10+