[HADOOP-1328] Hadoop Streaming needs to provide a way for the stream plugin to update global counters

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.18.0
• Hadoop Flags: Reviewed

Release Note

Introduced a way for a streaming process to update global counters and status using stderr stream to emit information. Use "reporter:counter:<group>,<counter>,<amount> " to update a counter. Use "reporter:status:<message>" to update status.

• Assignee: Thomas White
• Reporter: Runping Qi
• Created: 04/May/07 21:37
• Updated: 08/Jul/09 17:05
• Resolved: 04/Jun/08 20:21

Description

Sometimes, the stream plugin may want to create/update its own global counters.
There is no way to do so currently.
One possible way to enable that is to use the stderr output of the stream process.
The stream process can emit the global counter update information (such as GLOCAL_COUNTER COUNTER_NAME NUM)
to the stderr of the process. The Stderr handling thread can call the global counter API whenever it encounters the global counter update information line.

Issue Links

• is blocked by: HADOOP-1915 adding counters methods using String (as opposed to Enum)