[FLUME-2748] ThriftLegacySource produces exception due to wrongly compiled thrift definitions

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.6.0
• Component/s: Sinks+Sources
• Assignee: Unassigned
• Reporter: Tobias Heintz
• Created: 27/Jul/15 17:07
• Updated: 27/Jul/15 17:07

Description

We are in the process of upgrading our Flume installation from 0.9.2 to 1.6.0. Currently we are using the ThriftLegacySource to allow the Flume server to receive messages without having to update all components at the same time. For every received message, we are seeing this exception:

2015-07-24 17:15:28,892 (pool-3-thread-5) [ERROR - org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:215)] Error occurred during processing of message.

java.lang.NullPointerException
	at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:53)
	at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
	at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)

I've done some digging in the code and it appears that there is an error in the Java classes that were compiled from the legacy thrift definitions: the method append is defined as oneway, however in the compiled class, the method isOneway() returns false. This then leads to the NullPointerException, when the ProcessFunction tries to write the result back to the producer.

I'm not sure how this happened, maybe the very old version (0.7) of the thrift compiler is at fault here. The fix however would be to simply make the isOneway() method return true.