[ARROW-17303] [Java] Read "arrow" (IPC and streaming) files using org.apache.arrow.dataset.jni.NativeDatasetFactory

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 9.0.0
• Fix Version/s: 10.0.0
• Component/s: Java
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/32580
• Assignee: Igor Suhorukov
• Reporter: Igor Suhorukov
• Created: 04/Aug/22 08:55
• Updated: 11/Jan/23 11:49
• Resolved: 08/Aug/22 19:45

Description

Fetch "arrow" (IPC and streaming) files using org.apache.arrow.dataset.jni.NativeDatasetFactory in Java API.  This functionality required to implement Arrow file/Stream input format in my use case to process large amount of existing geospatial ARROW format data in Apache Spark data source. Optimized Analytics Package (OAP) for Spark also can leverage this feature of Dataset on JVM. They use FileSystemDatasetFactory in this [Spark gazelle_plugin adapter

Issue Links

• links to: GitHub Pull Request #13811
• links to: Read "arrow" (IPC and streaming) files in Java Dataset API