Flume
  1. Flume
  2. FLUME-2027

Check for default replication fails on federated cluster in hdfs sink

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: v1.4.0
    • Component/s: None
    • Labels:
      None

      Description

      Seeing the following exception when calling fs.getDefaultReplication() on a federated fs:

      13/04/26 08:52:12 ERROR hdfs.HDFSEventSink: process failed
      org.apache.hadoop.fs.viewfs.NotInMountpointException:
      getDefaultReplication on empty path is invalid
              at org.apache.hadoop.fs.viewfs.ViewFileSystem.getDefaultReplication(ViewFileSystem.java:509)
              at org.apache.flume.sink.hdfs.AbstractHDFSWriter.getFsDesiredReplication(AbstractHDFSWriter.java:104)
              at org.apache.flume.sink.hdfs.AbstractHDFSWriter.isUnderReplicated(AbstractHDFSWriter.java:73)
              at org.apache.flume.sink.hdfs.BucketWriter.shouldRotate(BucketWriter.java:486)
              at org.apache.flume.sink.hdfs.BucketWriter.append(BucketWriter.java:421)
              at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:392)
              at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
              at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
              at java.lang.Thread.run(Thread.java:722)
      
      1. FLUME-2027-1.patch
        6 kB
        Mike Percy
      2. FLUME-2027.patch
        5 kB
        Mike Percy

        Issue Links

          Activity

          Hari Shreedharan made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Mike Percy made changes -
          Attachment FLUME-2027-1.patch [ 12581345 ]
          Mike Percy made changes -
          Remote Link This issue links to "Review Board (Web Link)" [ 12183 ]
          Mike Percy made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Fix Version/s v1.4.0 [ 12323372 ]
          Mike Percy made changes -
          Attachment FLUME-2027.patch [ 12580947 ]
          Mike Percy made changes -
          Assignee Mike Percy [ mpercy ]
          Mike Percy made changes -
          Link This issue relates to HBASE-6067 [ HBASE-6067 ]
          Mike Percy made changes -
          Field Original Value New Value
          Link This issue relates to HADOOP-8422 [ HADOOP-8422 ]
          Mike Percy created issue -

            People

            • Assignee:
              Mike Percy
              Reporter:
              Mike Percy
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development