Trivial Description
I was having the following error starting flume-master with the Debian init script and was stumped diagnosing it:
2010-07-09 23:41:51,651 INFO com.cloudera.flume.watchdog.Watchdog: Dropped a pidfile='/tmp/flumemaster.pid' with pid=3374
2010-07-09 23:41:51,719 INFO com.cloudera.flume.conf.FlumeConfiguration: Loading configurations from /etc/flume/conf
2010-07-09 23:41:51,867 INFO com.cloudera.flume.watchdog.Watchdog: Restarting process @ Fri Jul 09 23:41:51 UTC 2010
2010-07-09 23:41:52,294 INFO com.cloudera.flume.master.FlumeMaster: Flume 0.9.0+1
2010-07-09 23:41:52,294 INFO com.cloudera.flume.master.FlumeMaster: rev 7fde7f12540ae30a2d5396115c7fa343a66fd56d
2010-07-09 23:41:52,294 INFO com.cloudera.flume.master.FlumeMaster: Compiled on Mon Jun 28 13:39:04 PDT 2010
2010-07-09 23:41:52,373 INFO com.cloudera.flume.conf.FlumeConfiguration: Loading configurations from /etc/flume/conf
2010-07-09 23:41:52,651 INFO com.cloudera.flume.master.FlumeMaster: Starting flume master on: domU-12-31-38-04-7C-68.compute-1.internal
2010-07-09 23:41:52,651 INFO com.cloudera.flume.master.FlumeMaster: Working Directory is: /usr/lib/flume/.
2010-07-09 23:41:52,651 INFO com.cloudera.flume.master.ZooKeeperService: Starting standalone ZooKeeper instance on port 2181
2010-07-09 23:41:52,655 ERROR com.cloudera.flume.master.FlumeMaster: IO problem: Permission denied
2010-07-09 23:41:52,667 INFO com.cloudera.flume.watchdog.Watchdog: Subprocess exited with value 0
2010-07-09 23:41:52,667 INFO com.cloudera.flume.watchdog.Watchdog: Subprocess exited cleanly, closing watchdog
2010-07-09 23:41:52,667 INFO com.cloudera.flume.watchdog.Watchdog: Watchdog shutdown hook
I eventually realized the cause was that I was following the user guide, and had initially started the flume master by typing "flume master" as root. The file /tmp/flume-zk/server-0/myid was created by root, and thus could not be deleted by the flume user. To figure this out I had to su -s /bin/sh flume, then run "flume master" to see the filename causing the problem. It would have been helpful if the /var/log/flume entry would have said
2010-07-09 23:41:52,655 ERROR com.cloudera.flume.master.FlumeMaster: IO problem: Permission denied on file /tmp/flume-zk/server-0/myid
instead of just
2010-07-09 23:41:52,655 ERROR com.cloudera.flume.master.FlumeMaster: IO problem: Permission denied
No biggie though, filed at tlipcon's request! Great job guys!
-Ben Standefer