[AVRO-2926] API can't guarantee objects are not None

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: python
• Assignee: Unassigned
• Reporter: Michael A. Smith
• Created: 22/Aug/20 01:14
• Updated: 22/Aug/20 01:14

Description

Several objects in the Python API can be constructed with optional arguments. Nothing in their code guarantees those arguments are reified or even aligned. A DataFileWriter, for example, requires a DatumWriter, but does not require a schema. A DatumWriter also does not require a schema. So you can attempt to construct and use a DatumWriter or a DataFileWriter whose schema is always None.

This and similar bugs are found as I add type hints to these classes. Their types cannot be improved without breaking changes.

I would like to change the API to avoid this kind of pitfall. In this case, I would like for DataFileWriter to require a writers_schema and to construct its own DatumWriter using that schema. I would like DatumWriter to require a schema as well, and for both of these classes to not allow their schemas to be changed after construction.

This is technically an API breaking change, but I believe the API is too fragile today.