[CHUKWA-282] Demux should output detailed per-operation ClientTrace records

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.2.0
• Fix Version/s: 0.2.0
• Component/s: Data Processors
• Assignee: Jiaqi Tan
• Reporter: Jiaqi Tan
• Created: 09/Jun/09 17:45
• Updated: 12/Jul/09 21:15
• Resolved: 12/Jul/09 21:15

Description

Currently, the ClientTrace reducer in Demux summarizes client trace operations to produce aggregate statistics for each type of operation (e.g. total bytes of local HDFS reads, total bytes of local HDFS writes). However, more sophisticated algorithms and visualization will need more detailed information such as amount of reads/writes to/from every pair of hosts. The ClientTrace reducer should also output the unsummarized, raw ClientTrace data so they can be used for analysis/visualization.