[ARROW-15618] [C++] Correctly handle Substrait nullability of types

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: C++
• Labels: substrait
• External issue URL: https://github.com/apache/arrow/issues/20108

Language

C++

• Assignee: Unassigned
• Reporter: Jeroen van Straten
• Created: 08/Feb/22 18:53
• Updated: 11/Jan/23 11:38

Description

In Substrait, nullability is associated with a type, whereas in Arrow it is associated with a field. Currently, DeserializeType ignores nullability for this reason, and SerializeType uses the (AFAICT) illegal NULLABILITY_UNSPECIFIED enum variant to do round-trip testing, and therefore this illegal variant is not rejected as it should.

The correct solution IMO is to (de)serialize types to and from std::pair<std::shared_ptr<DataType>, bool> to explicitly expose this difference to the outside world. The internal FromProto and ToProto are functions already doing this.