[FLUME-842] NPE when using jdbc channel with 2 local nodes communicating through avro

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: NG alpha 1
• Fix Version/s: notrack
• Component/s: Channel

Environment

flume-728 branch, commit object 0014756b8462ed2711d2e3af4ccabcabff4fdc70

Props file "flume.properties":
host1.channels = jdbcChannel
host1.sources = avroSource
host1.sinks = avroSink
#
host1.channels.jdbcChannel.type=jdbc
#
host1.sources.avroSource.type=avro
host1.sources.avroSource.channels=jdbcChannel
host1.sources.avroSource.bind=0.0.0.0
host1.sources.avroSource.port=41414
#
host1.sinks.avroSink.type=avro
host1.sinks.avroSink.channel=jdbcChannel
host1.sinks.avroSink.hostname=localhost
host1.sinks.avroSink.port=41415
host1.sinks.avroSink.batch-size=100
#-----
host2.channels=jdbcChannel
host2.sources=avroSource
host2.sinks=loggerSink
#
host2.channels.jdbcChannel.type=jdbc
#
host2.sources.avroSource.type=avro
host2.sources.avroSource.channels=jdbcChannel
host2.sources.avroSource.bind=0.0.0.0
host2.sources.avroSource.port=41415
#
host2.sinks.loggerSink.type=logger
host2.sinks.loggerSink.channel=jdbcChannel

• Assignee: Arvind Prabhakar
• Reporter: Will McQueen
• Created: 07/Nov/11 19:15
• Updated: 14/Mar/12 16:37
• Resolved: 22/Nov/11 01:23

Description

Expected comm looks like this:
AvroCLIClient ==(41414)==> avroSource=>jdbcChannel=>avroSink ==(41415)==> avroSource=>jdbcChannel=>loggerSink

Shorthand:
AvroCLIClient ==(41414)==> host1 ==(41415)==> host2

Steps:
1) Run host2 with main class org.apache.flume.node.Application and args:
-f /home/will/git/apache/flume/conf/flume.properties -n host2
2) Run host1 with same main class, and args:
-f /home/will/git/apache/flume/conf/flume.properties -n host1
3) Run Avro client with main class org.apache.flume.client.avro.AvroCLIClient and args:
-H localhost -p 41414 -F /etc/passwd

I see the following NPE on host1 after step 2 (and before step 3). There's no problem when switching to memory channel for both hosts.

2011-11-07 11:00:42,103 (conf-file-poller-0) [ERROR - org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:205)] Failed to load configuration data. Exception follows.
java.lang.NullPointerException
at org.apache.flume.channel.jdbc.impl.DerbySchemaHandler.schemaExists(DerbySchemaHandler.java:321)
at org.apache.flume.channel.jdbc.impl.JdbcChannelProviderImpl.initializeSchema(JdbcChannelProviderImpl.java:108)
at org.apache.flume.channel.jdbc.impl.JdbcChannelProviderImpl.initialize(JdbcChannelProviderImpl.java:95)
at org.apache.flume.channel.jdbc.JdbcChannelProviderFactory.getProvider(JdbcChannelProviderFactory.java:35)
at org.apache.flume.channel.jdbc.JdbcChannel.configure(JdbcChannel.java:81)
at org.apache.flume.conf.Configurables.configure(Configurables.java:22)
at org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.loadChannels(PropertiesFileConfigurationProvider.java:223)
at org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.load(PropertiesFileConfigurationProvider.java:184)
at org.apache.flume.conf.file.AbstractFileConfigurationProvider.doLoad(AbstractFileConfigurationProvider.java:124)
at org.apache.flume.conf.file.AbstractFileConfigurationProvider.access$1(AbstractFileConfigurationProvider.java:114)
at org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:203)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)