Affects Version/s: None
Fix Version/s: None
It seems that FILE_BYTES_READ/FILE_BYTES_WRITTEN can NOT be updated in streaming application by:
The syntax I tried is:
System.err.println("reporter:counter:"+ "FileSystemCounters" + ","
+ "FILE_BYTES_READ" + ","
Since File System counters are added later into the framework by ticket
and that's why it is NOT properly handled by streaming.jar.
The implementation of File System counter is a bit different, and it keeps track of the real information in FileSystem.Statistics object. When the task finished, the framework would read the info from 'FileSystem.Statistics' and set the File System counters accordingly.
Therefore, the regular way to update counters – ' findCounter() and Counter.incrment() ' would NOT work for the File System counters as they would be overwritten by the info in 'FileSystem.Statistics ' eventually.
However, it seems that the 'reporter:counter:' scheme in streaming.jar is implemented by the regular way, and maybe that's why it is not working.