[FLUME-2027] Check for default replication fails on federated cluster in hdfs sink

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.4.0
• Assignee: Mike Percy
• Reporter: Mike Percy
• Created: 26/Apr/13 19:01
• Updated: 07/May/13 09:24
• Resolved: 07/May/13 03:48

Description

Seeing the following exception when calling fs.getDefaultReplication() on a federated fs:

13/04/26 08:52:12 ERROR hdfs.HDFSEventSink: process failed
org.apache.hadoop.fs.viewfs.NotInMountpointException:
getDefaultReplication on empty path is invalid
        at org.apache.hadoop.fs.viewfs.ViewFileSystem.getDefaultReplication(ViewFileSystem.java:509)
        at org.apache.flume.sink.hdfs.AbstractHDFSWriter.getFsDesiredReplication(AbstractHDFSWriter.java:104)
        at org.apache.flume.sink.hdfs.AbstractHDFSWriter.isUnderReplicated(AbstractHDFSWriter.java:73)
        at org.apache.flume.sink.hdfs.BucketWriter.shouldRotate(BucketWriter.java:486)
        at org.apache.flume.sink.hdfs.BucketWriter.append(BucketWriter.java:421)
        at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:392)
        at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
        at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
        at java.lang.Thread.run(Thread.java:722)

Issue Links

• relates to: HADOOP-8422 Deprecate FileSystem#getDefault* and getServerDefault methods that don't take a Path argument
• relates to: HBASE-6067 HBase won't start when hbase.rootdir uses ViewFileSystem
• links to: Review Board