Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-2027

Check for default replication fails on federated cluster in hdfs sink

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.4.0
    • Component/s: None
    • Labels:
      None

      Description

      Seeing the following exception when calling fs.getDefaultReplication() on a federated fs:

      13/04/26 08:52:12 ERROR hdfs.HDFSEventSink: process failed
      org.apache.hadoop.fs.viewfs.NotInMountpointException:
      getDefaultReplication on empty path is invalid
              at org.apache.hadoop.fs.viewfs.ViewFileSystem.getDefaultReplication(ViewFileSystem.java:509)
              at org.apache.flume.sink.hdfs.AbstractHDFSWriter.getFsDesiredReplication(AbstractHDFSWriter.java:104)
              at org.apache.flume.sink.hdfs.AbstractHDFSWriter.isUnderReplicated(AbstractHDFSWriter.java:73)
              at org.apache.flume.sink.hdfs.BucketWriter.shouldRotate(BucketWriter.java:486)
              at org.apache.flume.sink.hdfs.BucketWriter.append(BucketWriter.java:421)
              at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:392)
              at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
              at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
              at java.lang.Thread.run(Thread.java:722)
      

        Attachments

        1. FLUME-2027-1.patch
          6 kB
          Mike Percy
        2. FLUME-2027.patch
          5 kB
          Mike Percy

          Issue Links

            Activity

              People

              • Assignee:
                mpercy Mike Percy
                Reporter:
                mpercy Mike Percy
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: