Details
-
Bug
-
Status: Closed
-
Minor
-
Resolution: Won't Fix
-
3.4.0
-
None
-
openjdk version "11.0.19" 2023-04-18
OpenJDK Runtime Environment (build 11.0.19+7-post-Ubuntu-0ubuntu122.04.1)
OpenJDK 64-Bit Server VM (build 11.0.19+7-post-Ubuntu-0ubuntu122.04.1, mixed mode, sharing)
Description
I've consistently observed the following error when compiling several DFDL schemas concurrently on OpenJDK 11 and 17:
Caused by: org.apache.daffodil.dsom.SchemaDefinitionError: Schema Definition Error: Unsupported encoding: ISO-8859-1. Supported encodings: Schema context: E1153 Location line 1524 column 14 in file:/home/smooks-edi-cartridge/edifact-schemas/target/classes/d94b/EDIFACT-Segments.dfdl.xsd at org.apache.daffodil.dsom.ImplementsThrowsSDE.SDE (SDE.scala:179) at org.apache.daffodil.dsom.ImplementsThrowsSDE.SDE$ (SDE.scala:178) at org.apache.daffodil.dsom.DPathCompileInfo.SDE (CompiledExpression1.scala:210) at org.apache.daffodil.exceptions.ThrowsSDE.schemaDefinitionError (ThrowsSDE.scala:58) at org.apache.daffodil.exceptions.ThrowsSDE.schemaDefinitionError$ (ThrowsSDE.scala:58) at org.apache.daffodil.dsom.DPathCompileInfo.schemaDefinitionError (CompiledExpression1.scala:210) at org.apache.daffodil.processors.CharsetEvBase.compute (EvEncoding.scala:99) at org.apache.daffodil.processors.CharsetEvBase.compute (EvEncoding.scala:81) at org.apache.daffodil.processors.Evaluatable.evaluate (Evaluatable.scala:281) at org.apache.daffodil.processors.Evaluatable.compileTimeEvaluate (Evaluatable.scala:212) at org.apache.daffodil.processors.Evaluatable.compile (Evaluatable.scala:317) at org.apache.daffodil.processors.Evaluatable.compile (Evaluatable.scala:326) at org.apache.daffodil.dsom.TermRuntimeValuedPropertiesMixin.maybeCharsetEv (RuntimePropertyMixins.scala:122) at org.apache.daffodil.dsom.TermRuntimeValuedPropertiesMixin.maybeCharsetEv$ (RuntimePropertyMixins.scala:119) at org.apache.daffodil.dsom.LocalElementDeclBase.maybeCharsetEv$lzycompute (LocalElementDecl.scala:25) at org.apache.daffodil.dsom.LocalElementDeclBase.maybeCharsetEv (LocalElementDecl.scala:25) at org.apache.daffodil.dsom.TermRuntimeValuedPropertiesMixin.charsetEv (RuntimePropertyMixins.scala:115) at org.apache.daffodil.dsom.TermRuntimeValuedPropertiesMixin.charsetEv$ (RuntimePropertyMixins.scala:114) at org.apache.daffodil.dsom.LocalElementDeclBase.charsetEv$lzycompute (LocalElementDecl.scala:25) at org.apache.daffodil.dsom.LocalElementDeclBase.charsetEv (LocalElementDecl.scala:25) at org.apache.daffodil.dsom.TermEncodingMixin.encodingInfo (TermEncodingMixin.scala:104) at org.apache.daffodil.dsom.TermEncodingMixin.encodingInfo$ (TermEncodingMixin.scala:102) at org.apache.daffodil.dsom.LocalElementDeclBase.encodingInfo$lzycompute (LocalElementDecl.scala:25) at org.apache.daffodil.dsom.LocalElementDeclBase.encodingInfo (LocalElementDecl.scala:25) at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$elementRuntimeData$1 (ElementBaseRuntime1Mixin.scala:180) at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1 (OOLAG.scala:679) at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute (OOLAG.scala:677) at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny (OOLAG.scala:674) at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute (OOLAG.scala:724) at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value (OOLAG.scala:724) at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData (ElementBaseRuntime1Mixin.scala:174) at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData$ (ElementBaseRuntime1Mixin.scala:174) at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData$lzycompute (LocalElementDecl.scala:25) at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData (LocalElementDecl.scala:25) at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$childrenERDs$2 (ElementBaseRuntime1Mixin.scala:170) at scala.collection.immutable.List.map (List.scala:293) at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$childrenERDs$1 (ElementBaseRuntime1Mixin.scala:169) at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1 (OOLAG.scala:679) at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute (OOLAG.scala:677) at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny (OOLAG.scala:674) at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute (OOLAG.scala:724) at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value (OOLAG.scala:724) at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs (ElementBaseRuntime1Mixin.scala:169) at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs$ (ElementBaseRuntime1Mixin.scala:168) at org.apache.daffodil.dsom.LocalElementDeclBase.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs$lzycompute (LocalElementDecl.scala:25) at org.apache.daffodil.dsom.LocalElementDeclBase.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs (LocalElementDecl.scala:25) at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$elementRuntimeData$1 (ElementBaseRuntime1Mixin.scala:177) at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1 (OOLAG.scala:679) at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute (OOLAG.scala:677) at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny (OOLAG.scala:674) at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute (OOLAG.scala:724) at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value (OOLAG.scala:724) at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData (ElementBaseRuntime1Mixin.scala:174) at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData$ (ElementBaseRuntime1Mixin.scala:174) at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData$lzycompute (LocalElementDecl.scala:25) at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData (LocalElementDecl.scala:25) at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$childrenERDs$2 (ElementBaseRuntime1Mixin.scala:170) at scala.collection.immutable.List.map (List.scala:293) at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$childrenERDs$1 (ElementBaseRuntime1Mixin.scala:169) at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1 (OOLAG.scala:679) at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute (OOLAG.scala:677) at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny (OOLAG.scala:674) at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute (OOLAG.scala:724) at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value (OOLAG.scala:724) at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs (ElementBaseRuntime1Mixin.scala:169) at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs$ (ElementBaseRuntime1Mixin.scala:168) at org.apache.daffodil.dsom.LocalElementDeclBase.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs$lzycompute (LocalElementDecl.scala:25) at org.apache.daffodil.dsom.LocalElementDeclBase.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs (LocalElementDecl.scala:25) at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$elementRuntimeData$1 (ElementBaseRuntime1Mixin.scala:177) at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1 (OOLAG.scala:679) at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute (OOLAG.scala:677) at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny (OOLAG.scala:674) at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute (OOLAG.scala:724) at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value (OOLAG.scala:724) at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData (ElementBaseRuntime1Mixin.scala:174) at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData$ (ElementBaseRuntime1Mixin.scala:174) at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData$lzycompute (LocalElementDecl.scala:25) at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData (LocalElementDecl.scala:25) at org.apache.daffodil.runtime1.LocalElementDeclBaseRuntime1Mixin.$anonfun$$init$$1 (LocalElementDeclBaseRuntime1Mixin.scala:24) at scala.runtime.java8.JFunction0$mcV$sp.apply (JFunction0$mcV$sp.java:23) at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1 (OOLAG.scala:679) at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute (OOLAG.scala:677) at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny (OOLAG.scala:674) at org.apache.daffodil.oolag.OOLAG$OOLAGHost.$anonfun$checkErrors$2 (OOLAG.scala:427) at scala.runtime.java8.JFunction0$mcV$sp.apply (JFunction0$mcV$sp.java:23) at org.apache.daffodil.oolag.OOLAG$.keepGoing (OOLAG.scala:65) at org.apache.daffodil.oolag.OOLAG$OOLAGHost.checkErrors (OOLAG.scala:427) at org.apache.daffodil.oolag.OOLAG$OOLAGHost.checkErrors$ (OOLAG.scala:413) at org.apache.daffodil.dsom.SchemaComponentImpl.checkErrors (SchemaComponent.scala:34) at org.apache.daffodil.oolag.OOLAG$OOLAGHost.isError (OOLAG.scala:484) at org.apache.daffodil.oolag.OOLAG$OOLAGHost.isError$ (OOLAG.scala:483) at org.apache.daffodil.dsom.SchemaSet.isError (SchemaSet.scala:543) at org.apache.daffodil.compiler.ProcessorFactory.isError (Compiler.scala:136) at org.apache.daffodil.compiler.Compiler.org$apache$daffodil$compiler$Compiler$$compileSourceInternal (Compiler.scala:419) at org.apache.daffodil.compiler.Compiler$.org$apache$daffodil$compiler$Compiler$$compileSourceSynchronizer (Compiler.scala:461) at org.apache.daffodil.compiler.Compiler.compileSource (Compiler.scala:404) at org.apache.daffodil.compiler.Compiler.compileFile (Compiler.scala:388) at org.apache.daffodil.japi.Compiler.compileFile (Daffodil.scala:146) at org.apache.daffodil.japi.Compiler.compileFile (Daffodil.scala:126) at org.smooks.edi.edg.EdifactDfdlSchemaGenerator.generateDfdlSchemas (EdifactDfdlSchemaGenerator.java:141)
Precisely, the error occurs on the line of code:
Daffodil.compiler().compileFile(...)
The error doesn't always occur on the same DFDL schema or the same schema location line for that matter. Curiously, this error, which I presume is happening because of a race-condition, doesn't show up on OpenJDK 8 so it could be a bug in the Java runtime. Note that a new Daffodil compiler is created to compile each schema. Moving Daffodil.compiler().compileFile(...) inside a synchronized block doesn't make a difference.
I had a hunch and added the following code before forking the thread to concurrently compile the schemas:
org.apache.daffodil.processors.charset.CharsetUtils.supportedEncodingsString()
The above makes the error go away and I'm using it as a workaround. Can't really say why so but I suspect the race-condition is caused by cache reading that takes place when checking whether the encoding exists during schema compilation.
I've created a branch where you should be able to reproduce the problem. Steps to reproduce:
- git clone https://github.com/claudemamo/smooks-edi-cartridge/tree/daffodil-unsupported-encoding-issue --branch daffodil-unsupported-encoding-issue
- cd smooks-edi-cartridge
- mvn clean install -DskipTests
- The build should fail on the edifact-schemas Maven module because of this issue. The error should occur at this point: https://github.com/claudemamo/smooks-edi-cartridge/blob/daffodil-unsupported-encoding-issue/edg/src/main/java/org/smooks/edi/edg/EdifactDfdlSchemaGenerator.java#L139. Uncomment the following line to apply the workaround: https://github.com/claudemamo/smooks-edi-cartridge/blob/daffodil-unsupported-encoding-issue/edg/src/main/java/org/smooks/edi/edg/EdifactDfdlSchemaGenerator.java#L95