[STORM-2265] Incorrectly Serialized JSON in TransactionalState causes Worker to Die

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Won't Fix
• Affects Version/s: 2.0.0, 1.1.0, 1.0.3
• Component/s: storm-core
• Assignee: Hugo Da Cruz Louro
• Reporter: Hugo Da Cruz Louro
• Created: 28/Dec/16 15:43
• Updated: 30/Aug/17 05:45
• Resolved: 30/Aug/17 05:45

Description

TransactionalState uses JSONValue to serialize / deserialize objects. However, the object GlobalPartitionInformation is incorrectly serialized by default, causing the exception bellow. To get around this problem, GlobalPartitionInformation must implement JSONAware.

2016-12-23 14:37:26.980 o.a.s.e.e.ReportError Thread-21-$spoutcoord-spout-spout1-executor[2, 2] [ERROR] Error
java.lang.RuntimeException: java.lang.RuntimeException: Unexpected character (G) at position 1.
at org.apache.storm.utils.Utils$6.run(Utils.java:2190) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
Caused by: java.lang.RuntimeException: Unexpected character (G) at position 1.
at org.apache.storm.trident.topology.state.TransactionalState.getData(TransactionalState.java:174) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
at org.apache.storm.trident.topology.state.RotatingTransactionalState.sync(RotatingTransactionalState.java:165) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
at org.apache.storm.trident.topology.state.RotatingTransactionalState.<init>(RotatingTransactionalState.java:46) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
at org.apache.storm.trident.spout.TridentSpoutCoordinator.prepare(TridentSpoutCoordinator.java:57) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
at org.apache.storm.topology.BasicBoltExecutor.prepare(BasicBoltExecutor.java:43) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
at org.apache.storm.executor.bolt.BoltExecutor.init(BoltExecutor.java:84) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
at org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:93) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
at org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:45) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
at org.apache.storm.utils.Utils$6.run(Utils.java:2179) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
... 1 more
Caused by: org.apache.storm.shade.org.json.simple.parser.ParseException
at org.apache.storm.shade.org.json.simple.parser.Yylex.yylex(Unknown Source) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
at org.apache.storm.shade.org.json.simple.parser.JSONParser.nextToken(Unknown Source) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
at org.apache.storm.shade.org.json.simple.parser.JSONParser.parse(Unknown Source) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
at org.apache.storm.shade.org.json.simple.parser.JSONParser.parse(Unknown Source) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
at org.apache.storm.shade.org.json.simple.parser.JSONParser.parse(Unknown Source) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
at org.apache.storm.shade.org.json.simple.JSONValue.parseWithException(Unknown Source) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
at org.apache.storm.trident.topology.state.TransactionalState.getData(TransactionalState.java:167) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
at org.apache.storm.trident.topology.state.RotatingTransactionalState.sync(RotatingTransactionalState.java:165) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
at org.apache.storm.trident.topology.state.RotatingTransactionalState.<init>(RotatingTransactionalState.java:46) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
at org.apache.storm.trident.spout.TridentSpoutCoordinator.prepare(TridentSpoutCoordinator.java:57) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
at org.apache.storm.topology.BasicBoltExecutor.prepare(BasicBoltExecutor.java:43) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
at org.apache.storm.executor.bolt.BoltExecutor.init(BoltExecutor.java:84) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
at org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:93) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
at org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:45) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
at org.apache.storm.utils.Utils$6.run(Utils.java:2179) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
... 1 more
2016-12-23 14:37:26.987 o.a.s.u.Utils Thread-21-$spoutcoord-spout-spout1-executor[2, 2] [ERROR] Halting process: Worker died
java.lang.RuntimeException: Halting process: Worker died
at org.apache.storm.utils.Utils.exitProcess(Utils.java:1792) [storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
at org.apache.storm.utils.Utils$4.run(Utils.java:1800) [storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
at org.apache.storm.executor.error.ReportErrorAndDie.uncaughtException(ReportErrorAndDie.java:45) [storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
at java.lang.Thread.dispatchUncaughtException(Thread.java:1956) [?:1.8.0_112]
2016-12-23 14:37:26.987 o.a.s.d.w.Worker Thread-38 [INFO] Shutting down worker tkst-consumer-4-1482532570 556a1e7b-49f7-4dc2-a936-d17e5e4ba9de 6700
2016-12-23 14:37:26.988 o.a.s.d.w.Worker Thread-38 [INFO] Terminating messaging context

Issue Links

• duplicates: STORM-2264 OpaqueTridentKafkaSpout failing after STORM-2216
• links to: GitHub Pull Request #1844