Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-2027

Check for default replication fails on federated cluster in hdfs sink

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.4.0
    • None
    • None

    Description

      Seeing the following exception when calling fs.getDefaultReplication() on a federated fs:

      13/04/26 08:52:12 ERROR hdfs.HDFSEventSink: process failed
      org.apache.hadoop.fs.viewfs.NotInMountpointException:
      getDefaultReplication on empty path is invalid
              at org.apache.hadoop.fs.viewfs.ViewFileSystem.getDefaultReplication(ViewFileSystem.java:509)
              at org.apache.flume.sink.hdfs.AbstractHDFSWriter.getFsDesiredReplication(AbstractHDFSWriter.java:104)
              at org.apache.flume.sink.hdfs.AbstractHDFSWriter.isUnderReplicated(AbstractHDFSWriter.java:73)
              at org.apache.flume.sink.hdfs.BucketWriter.shouldRotate(BucketWriter.java:486)
              at org.apache.flume.sink.hdfs.BucketWriter.append(BucketWriter.java:421)
              at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:392)
              at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
              at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
              at java.lang.Thread.run(Thread.java:722)
      

      Attachments

        1. FLUME-2027-1.patch
          6 kB
          Mike Percy
        2. FLUME-2027.patch
          5 kB
          Mike Percy

        Issue Links

          Activity

            People

              mpercy Mike Percy
              mpercy Mike Percy
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: