[NUTCH-2519] Log mapreduce job counters in local mode

• Type: Improvement
• Status: Closed
• Priority: Trivial
• Resolution: Fixed
• Affects Version/s: 2.3.1, 1.14
• Fix Version/s: 2.4, 1.15
• Patch Info: Patch Available
• Assignee: Sebastian Nagel
• Reporter: Sebastian Nagel
• Created: 05/Mar/18 12:26
• Updated: 01/Oct/19 14:29
• Resolved: 06/Mar/18 08:20

Description

A simple change in the log4j.properties would make the Hadoop job counters appear in the hadoop.log also in local mode:

log4j.logger.org.apache.hadoop.mapreduce.Job=INFO

This may provide useful information for debugging, esp. if counters are not explicitly logged by tools (see @user). This would make the output also more similar to (pseudo)distributed mode (Nutch is called via hadoop jar) Job counters and progress info are obligatorily logged.

Issue Links

• links to: GitHub Pull Request #287