[HADOOP-10728] Metrics system for Windows Azure Storage Filesystem

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.7.0
• Component/s: tools
• Target Version/s: 2.7.0
• Hadoop Flags: Reviewed
• Assignee: Mike Liddell
• Reporter: Mike Liddell
• Created: 20/Jun/14 01:18
• Updated: 12/Sep/17 16:01
• Resolved: 18/Dec/14 00:18

Description

Add a metrics2 source for the Windows Azure Filesystem driver that was introduced with HADOOP-9629.

AzureFileSystemInstrumentation is the new MetricsSource.

AzureNativeFilesystemStore and NativeAzureFilesystem have been modified to record statistics and some machinery is added for the accumulation of 'rolling average' statistics.

Primary new code appears in org.apache.hadoop.fs.azure.metrics namespace.

Credits and history

Credit for this work goes to the early team: minwei, davidlao, lengningliu and stojanovic as well as multiple people who have taken over this work since then (hope I don't forget anyone): dexterb, Johannes Klein, ivanmi, Michael Rys, mostafae, brian_swan, mikelid, xifang, and chuanliu.

Issue Links

• depends upon: HADOOP-9559 When metrics system is restarted MBean names get incorrectly flagged as dupes
• is related to: HADOOP-14862 Metrics for AdlFileSystem
• relates to: HADOOP-9629 Support Windows Azure Storage - Blob as a file system in Hadoop