[HADOOP-18884] [ABFS] Support VectorIO in ABFS Input Stream

• Type: Sub-task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.3.9
• Component/s: fs/azure
• Assignee: Anmol Asrani
• Reporter: Steve Loughran
• Created: 07/Sep/23 16:39
• Updated: 27/Feb/24 11:09

Description

the hadoop vector IO APIs are supported in file;// and s3a://; there's a hive ORC patch for this and PARQUET-2171 adds it for parquet -after which all apps using the library with a matching hadoop version and the feature enabled will get a significant speedup.

abfs needs to support too, which needs support for parallel GET requests for different ranges