[FLUME-2553] Error while saving data to hbase java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.Increment.setWriteToWAL(Z)Lorg/apache/hadoop/hbase/client/Increment;

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 1.5.0.1
• Fix Version/s: 1.5.0.1

Environment

Centos 6.6

• Assignee: Unassigned
• Reporter: Oleksiy Sayankin
• Created: 14/Nov/14 15:46
• Updated: 14/Nov/14 19:09
• Resolved: 14/Nov/14 19:08

Description

1. Create hbase.conf file /root/hbase.conf:

hbase-agent.sources = tail
hbase-agent.sinks = sink1
hbase-agent.channels = ch1
hbase-agent.sources.tail.type = exec
hbase-agent.sources.tail.command = tail -F /root/demo.txt
hbase-agent.sources.tail.channels = ch1
hbase-agent.sinks.sink1.type = org.apache.flume.sink.hbase.HBaseSink
hbase-agent.sinks.sink1.channel = ch1
hbase-agent.sinks.sink1.table = demo
hbase-agent.sinks.sink1.columnFamily = cf
hbase-agent.sinks.sink1.serializer = org.apache.flume.sink.hbase.SimpleHbaseEventSerializer
hbase-agent.sinks.sink1.serializer.payloadColumn = col1
hbase-agent.sinks.sink1.serializer.keyType = timestamp
hbase-agent.sinks.sink1.serializer.rowPrefix = 1
hbase-agent.sinks.sink1.serializer.suffix = timestamp
hbase-agent.channels.ch1.type=memory

2. Create demo file: /root/demo.txt:
value1
value2
value3
value4
value5
value6

3. Create table demo in hbase-0.98

hbase shell
create 'demo', 'cf'

4. Start flume

flume-ng agent --conf-file /root/hbase.conf --name hbase-agent -Dflume.root.logger=INFO,console

Get error:

14/11/14 11:08:34 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
14/11/14 11:08:34 INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
14/11/14 11:08:34 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux
14/11/14 11:08:34 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64
14/11/14 11:08:34 INFO zookeeper.ZooKeeper: Client environment:os.version=2.6.32-504.el6.x86_64
14/11/14 11:08:34 INFO zookeeper.ZooKeeper: Client environment:user.name=root
14/11/14 11:08:34 INFO zookeeper.ZooKeeper: Client environment:user.home=/root
14/11/14 11:08:34 INFO zookeeper.ZooKeeper: Client environment:user.dir=/root
14/11/14 11:08:34 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:5181 sessionTimeout=90000 watcher=hconnection-0x52dc5605, quorum=localhost:5181, baseZNode=/hbase
14/11/14 11:08:34 INFO zookeeper.RecoverableZooKeeper: Process identifier=hconnection-0x52dc5605 connecting to ZooKeeper ensemble=localhost:5181
14/11/14 11:08:34 INFO zookeeper.Login: successfully logged in.
14/11/14 11:08:34 INFO client.ZooKeeperSaslClient: Client will use SIMPLE-SECURITY as SASL mechanism.
14/11/14 11:08:34 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/0:0:0:0:0:0:0:1:5181. Will attempt to SASL-authenticate using Login Context section 'Client_simple'
14/11/14 11:08:34 INFO zookeeper.ClientCnxn: Socket connection established to localhost/0:0:0:0:0:0:0:1:5181, initiating session
14/11/14 11:08:34 INFO zookeeper.ClientCnxn: Session establishment complete on server localhost/0:0:0:0:0:0:0:1:5181, sessionid = 0x149aeb28c60003b, negotiated timeout = 40000
14/11/14 11:08:34 INFO Configuration.deprecation: hadoop.native.lib is deprecated. Instead, use io.native.lib.available
14/11/14 11:08:35 INFO instrumentation.MonitoredCounterGroup: Monitored counter group for type: SINK, name: sink1: Successfully registered new MBean.
14/11/14 11:08:35 INFO instrumentation.MonitoredCounterGroup: Component type: SINK, name: sink1 started
14/11/14 11:08:39 ERROR hbase.HBaseSink: Failed to commit transaction.Transaction rolled back.
java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.Increment.setWriteToWAL(Z)Lorg/apache/hadoop/hbase/client/Increment;
at org.apache.flume.sink.hbase.HBaseSink$4.run(HBaseSink.java:408)
at org.apache.flume.sink.hbase.HBaseSink$4.run(HBaseSink.java:391)
at org.apache.flume.sink.hbase.HBaseSink.runPrivileged(HBaseSink.java:427)
at org.apache.flume.sink.hbase.HBaseSink.putEventsAndCommit(HBaseSink.java:391)
at org.apache.flume.sink.hbase.HBaseSink.process(HBaseSink.java:344)
at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:745)
14/11/14 11:08:39 ERROR hbase.HBaseSink: Failed to commit transaction.Transaction rolled back.
java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.Increment.setWriteToWAL(Z)Lorg/apache/hadoop/hbase/client/Increment;
at org.apache.flume.sink.hbase.HBaseSink$4.run(HBaseSink.java:408)
at org.apache.flume.sink.hbase.HBaseSink$4.run(HBaseSink.java:391)
at org.apache.flume.sink.hbase.HBaseSink.runPrivileged(HBaseSink.java:427)
at org.apache.flume.sink.hbase.HBaseSink.putEventsAndCommit(HBaseSink.java:391)
at org.apache.flume.sink.hbase.HBaseSink.process(HBaseSink.java:344)
at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:745)
Exception in thread "SinkRunner-PollingRunner-DefaultSinkProcessor" java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.Increment.setWriteToWAL(Z)Lorg/apache/hadoop/hbase/client/Increment;
at org.apache.flume.sink.hbase.HBaseSink$4.run(HBaseSink.java:408)
at org.apache.flume.sink.hbase.HBaseSink$4.run(HBaseSink.java:391)
at org.apache.flume.sink.hbase.HBaseSink.runPrivileged(HBaseSink.java:427)
at org.apache.flume.sink.hbase.HBaseSink.putEventsAndCommit(HBaseSink.java:391)
at org.apache.flume.sink.hbase.HBaseSink.process(HBaseSink.java:344)
at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:745)