[HDFS-9680] Doing a lsr against WebImageViewer is slow

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Unassigned
• Reporter: Haohui Mai
• Created: 21/Jan/16 23:43
• Updated: 22/Jan/16 00:04

Description

We have experienced a performance issue that doing lsr against the WebImageViewer.

For a fsimage that has around 140m files, it takes around ~35 minutes to do the lsr across the live cluster, but ~12 hours to do the same operation against the WebImageViewer.

I believe that the root cause is that WebImageViewer decodes the protobuf messages on-demand which creates a lot of GC pressure. It might be better to decode it at the very beginning.