Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-1110

HDFS Sink throws IllegalStateException when flume-daemon shuts down

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Cannot Reproduce
    • Affects Version/s: 1.1.0
    • Fix Version/s: None
    • Component/s: Sinks+Sources
    • Labels:
      None

      Description

      When using HDFS sink, if you shutdown the daemon (sudo /etc/init.d/flume-ng-node stop), then an IllegalStateException is shown in the logs (/var/log/flume-ng/flume.log).

      2012-04-06 10:44:19,912 ERROR hdfs.HDFSEventSink: Error calling org.apache.flume.sink.hdfs.HDFSEventSink$4@32091738
      java.lang.IllegalStateException: Shutdown in progress
      at java.lang.ApplicationShutdownHooks.add(ApplicationShutdownHooks.java:39)
      at java.lang.Runtime.addShutdownHook(Runtime.java:192)
      at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:1607)
      at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1579)
      at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:228)
      at org.apache.hadoop.fs.Path.getFileSystem(Path.java:183)
      at org.apache.flume.sink.hdfs.BucketWriter.renameBucket(BucketWriter.java:196)
      at org.apache.flume.sink.hdfs.BucketWriter.close(BucketWriter.java:122)
      at org.apache.flume.sink.hdfs.HDFSEventSink$4.call(HDFSEventSink.java:440)
      at org.apache.flume.sink.hdfs.HDFSEventSink$4.call(HDFSEventSink.java:436)
      at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
      at java.util.concurrent.FutureTask.run(FutureTask.java:138)
      at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
      at java.lang.Thread.run(Thread.java:662)
      2012-04-06 10:44:19,927 INFO source.SyslogTcpSource: Syslog TCP Source stopping...
      2012-04-06 10:44:19,927 INFO source.SyslogTcpSource: Metrics:{ name:null counters:

      {events.success=11002}

      }

        Attachments

        1. FLUME-1110.patch
          0.7 kB
          Prasad Mujumdar

          Issue Links

            Activity

              People

              • Assignee:
                prasadm Prasad Mujumdar
                Reporter:
                prasadm Prasad Mujumdar
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: