[ARROW-17066] [C++][Substrait] "ignore_unknown_fields" should be specified when converting JSON to binary

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 9.0.0
• Component/s: C++
• Labels: pull-request-available substrait
• External issue URL: https://github.com/apache/arrow/issues/20319
• Assignee: Vibhatha Lakmal Abeykoon
• Reporter: Richard Tia
• Created: 13/Jul/22 17:54
• Updated: 11/Jan/23 11:48
• Resolved: 22/Jul/22 09:13

Description

https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.util.json_util#JsonParseOptions

 

When converting a substrait JSON to binary, there are many unknown fields that may exist since substrait is being built every week. ignore_unknown_fields should be specified when doing this conversion.

 

This is resulting in frequent errors similar to this:

E   pyarrow.lib.ArrowInvalid: JsonToBinaryStream returned INVALID_ARGUMENT:(relations[0].root.input.sort.input.aggregate.measures[0].measure) arguments: Cannot find field.

pyarrow/error.pxi:100: ArrowInvalid 

Issue Links

• links to: GitHub Pull Request #13605