[THRIFT-5488] SystemError when using fast binary or compact protocol in python 3.10

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.15.0
• Fix Version/s: 0.16.0
• Component/s: Python - Library

Language

Python

• Assignee: Ali-Akber Saifee
• Reporter: Ali-Akber Saifee
• Created: 12/Dec/21 00:30
• Updated: 30/Aug/22 21:31
• Resolved: 30/Aug/22 21:31

Description

Up till python 3.9  a DeprecationWarning was raised when '#' formats were used without defining PY_SSIZE_T_CLEAN:

 

DeprecationWarning: PY_SSIZE_T_CLEAN will be required for '#' formats
  iprot._fast_decode(self, iprot, [self.__class__, self.thrift_spec]) 

 

Starting with python 3.10 this now becomes a runtime SystemError. Details of change: Python bug tracker 40943

This specifically gets exercised in the c python extension module in ProtocolBase<Impl>::readBytes (src/protocol.tcc) when falling back to the building function.

An easy way to reproduce the error is to pass an empty byte array to the TSerializer.deserialize method, for example:

  buf = TTransport.TMemoryBuffer()
  transport = TTransport.TBufferedTransportFactory().getTransport(buf)
  factory = TBinaryProtocolAcceleratedFactory(transport)
  TSerialization.deserialize(Message(), b'', factory) #  Message is a thrift struct. This raises a SystemError in python 3.10+

Issue Links

• links to: GitHub Pull Request #2487