[HADOOP-14112] Über-jira adl:// Azure Data Lake Phase I: Stabilization

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0-alpha2
• Fix Version/s: 2.9.0, 3.0.0
• Component/s: fs/adl
• Target Version/s: 2.8.3
• Assignee: John Zhuge
• Reporter: Steve Loughran
• Created: 23/Feb/17 15:42
• Updated: 04/Dec/17 15:03
• Resolved: 04/Dec/17 15:03

Description

Uber JIRA to track things needed for Azure Data Lake to be considered stable and adl:// ready for wide use.

Based on the experience with other object stores, the things which usually surface once a stabilizing FS is picked up and used are

• handling of many GB files, up and down, be it: efficiency of read, when the writes take place, file leakage, time for close() and filesystem shutdown
• resilience to transient failures
• reporting of problems/diagnostics
• security option tuning
• race conditions
• differences between implementation and what actual applications expect

Issue Links

• is cloned by: HADOOP-14764 Über-jira adl:// Azure Data Lake Phase II: Performance, Resilience and Testing