Major Description
If a schema sets dfdl:encodingErrorPolicy="error", then Daffodil outputs this warning:
[warning] Schema Definition Warning: dfdl:encodingErrorPolicy="error" is not yet implemented. The 'replace' value will be used.
The idea is we want to be portable with schemas written for IBM DFDL (which only supports "error" and not "replace"), so we support the "error" property value but use the our only supported "replace" behavior. In the common case where there are no encoding errors, the behavior will be exactly the same as IBM DFDL.
But if there is an encoding error, then Daffodil throws this assertion:
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!! Not Yet Implemented !!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
You are using a feature that is not yet implemented:
Some(Not yet implemented: dfdl:encodingErrorPolicy="error"
org.apache.daffodil.lib.exceptions.Assert$.nyi(Assert.scala:151)
org.apache.daffodil.io.processors.charset.BitsCharsetDecoder.decodeOneHandleMalformed(BitsCharsetDecoder.scala:122)
org.apache.daffodil.io.processors.charset.BitsCharsetDecoder.decode(BitsCharsetDecoder.scala:84)
org.apache.daffodil.io.InputSourceDataInputStreamCharIterator.fetch(InputSourceDataInputStream.scala:906))
So even though the warning says we are going to use "replace", we still clearly use "error", which results in an assertion failure that we don't support it.
I confirmed this occurs at least as far back as Daffodil 3.0.0.
Here is a schema that reproduces the issue:
<include schemaLocation="/org/apache/daffodil/xsd/DFDLGeneralFormatPortable.dfdl.xsd" /> <annotation> <appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format ref="ex:GeneralFormat"/> </appinfo> </annotation> <element name="test" type="xs:string" dfdl:lengthKind="delimited" dfdl:terminator="%NL;" dfdl:encoding="ASCII" />
Any file that non-ASCII characters followed by a new line should trigger this.