[FLUME-2443] org.apache.hadoop.fs.FSDataOutputStream.sync() is deprecated in hadoop 2.4

• Type: Dependency upgrade
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.5.0, 1.5.0.1, 1.6.0
• Fix Version/s: 1.6.0
• Component/s: Sinks+Sources, Technical Debt

Environment

Linux PPC / x86

• Assignee: Hari Shreedharan
• Reporter: Corentin Baron
• Created: 07/Aug/14 07:34
• Updated: 03/Feb/15 22:37
• Resolved: 03/Feb/15 21:34

Description

HDFS sink uses this method, which is deprecated in hadoop 2.4.1, and no longer present in the current hadoop developments:

java.lang.NoSuchMethodError: org/apache/hadoop/fs/FSDataOutputStream.sync()V
at org.apache.flume.sink.hdfs.HDFSDataStream.sync(HDFSDataStream.java:131)
at org.apache.flume.sink.hdfs.BucketWriter$6.call(BucketWriter.java:502)
at org.apache.flume.sink.hdfs.BucketWriter$6.call(BucketWriter.java:499)
at org.apache.flume.sink.hdfs.BucketWriter$9$1.run(BucketWriter.java:718)
at org.apache.flume.sink.hdfs.BucketWriter.runPrivileged(BucketWriter.java:183)
at org.apache.flume.sink.hdfs.BucketWriter.access$1700(BucketWriter.java:59)
at org.apache.flume.sink.hdfs.BucketWriter$9.call(BucketWriter.java:715)
at java.util.concurrent.FutureTask.run(FutureTask.java:273)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1176)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at java.lang.Thread.run(Thread.java:853)