[ARROW-854] [Format] Support sparse tensor

• Type: New Feature
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 0.12.0
• Component/s: Format
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/16461
• Assignee: Kenta Murata
• Reporter: Kouhei Sutou
• Created: 19/Apr/17 04:37
• Updated: 11/Jan/23 07:11
• Resolved: 09/Jan/19 22:18

Description

From http://mail-archives.apache.org/mod_mbox/arrow-dev/201704.mbox/%3CCAJPUwMBV5pkiXqY1-hsfP%3D2qgjQg-hMKZMwmQa3jKdDfxSS26A%40mail.gmail.com%3E

since we have metadata for tensors/ndarrays and support in C++, having analogous metadata for sparse matrices seems reasonable. Someone else asked me about this in person recently (related to machine learning applications).

While many Arrow implementations will be focused on the in-memory columnar / record batch data model, having these traditional scientific computing memory layouts available for shared memory IPC will open up the Arrow C/C++ libraries to more use cases. We wouldn't necessarily expect other Arrow implementations (like Java) to implement these, though, unless they needed them.

Issue Links

• links to: GitHub Pull Request #2546