Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-2027

Check for default replication fails on federated cluster in hdfs sink

Attach filesAttach ScreenshotVotersStop watchingWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.4.0
    • None
    • None

    Description

      Seeing the following exception when calling fs.getDefaultReplication() on a federated fs:

      13/04/26 08:52:12 ERROR hdfs.HDFSEventSink: process failed
      org.apache.hadoop.fs.viewfs.NotInMountpointException:
      getDefaultReplication on empty path is invalid
              at org.apache.hadoop.fs.viewfs.ViewFileSystem.getDefaultReplication(ViewFileSystem.java:509)
              at org.apache.flume.sink.hdfs.AbstractHDFSWriter.getFsDesiredReplication(AbstractHDFSWriter.java:104)
              at org.apache.flume.sink.hdfs.AbstractHDFSWriter.isUnderReplicated(AbstractHDFSWriter.java:73)
              at org.apache.flume.sink.hdfs.BucketWriter.shouldRotate(BucketWriter.java:486)
              at org.apache.flume.sink.hdfs.BucketWriter.append(BucketWriter.java:421)
              at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:392)
              at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
              at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
              at java.lang.Thread.run(Thread.java:722)
      

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            mpercy Mike Percy
            mpercy Mike Percy
            Votes:
            0 Vote for this issue
            Watchers:
            3 Stop watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment