Nutch commands log to nutch/logs/hadoop.logs by default

• Key: NUTCH-342
• Type: Bug
• Status: Open
• Priority: Minor
• Assignee: Unassigned
• Reporter: Chris Schneider
• Affects Version/s: 0.8

Description

If (by default) Nutch commands are going to send their output to a file named "hadoop.log", then it seems like the default location for this file should be the same location where Hadoop is putting its hadoop.log file (i.e., $HADOOP_LOG_DIR). Currently, if I set HADOOP_LOG_DIR to a special location (via hadoop-env.sh), this has no effect on where Nutch commands send their output.

Some would probably suggest that I could just set NUTCH_LOG_DIR to $HADOOP_LOG_DIR myself. I still think that it should be defaulted this way in the nutch script. However, I'm unaware of an elegant way to modify such Nutch environment variables anyway. The hadoop-env.sh file provides a convenient place to modify Hadoop environment variables, but doing the same for Nutch environment variables presumably requires you to modify .bash_profile or a similar user script file (which is the way I used to accomplish this kind of thing with Nutch 0.7).

Comments

[Chris Schneider]

Here's a patch that defaults NUTCH_LOG_DIR to $HADOOP_LOG_DIR and NUTCH_LOGFILE to $HADOOP_LOG_FILE.

[Chris Schneider]

I apologize for my confusion. I had been thinking that hadoop-env.sh was getting sourced when a Nutch command was run; it is not. Thus, $HADOOP_LOG_DIR and $HADOOP_LOG_FILE are not set when executing Nutch commands. For now, I think it makes most sense for me to set NUTCH_LOG_DIR and NUTCH_LOGFILE to the same locations as $HADOOP_LOG_DIR and $HADOOP_LOG_FILE via .bash_profile, etc. I consider this awkward, but am unsure about how best to address this design problem. I'm beginning to think that NUTCH_LOGFILE should default to something like "nutch-$USER-$COMMAND-`hostname`.log", which would seem more appropriate to find within the $NUTCH_HOME/logs directory.

We should cleanup logging in nutch in general asap!
The way things are configured by today is everything else than elegant or clean.