[ARROW-7700] [Rust] All array types should have iterators and FromIterator support.

• Type: Wish
• Status: Closed
• Priority: Major
• Resolution: Invalid
• Component/s: Rust
• Labels: Usability
• External issue URL: https://github.com/apache/arrow/issues/23943
• Assignee: Unassigned
• Reporter: Andy Thomason
• Created: 28/Jan/20 09:40
• Updated: 11/Jan/23 07:55
• Resolved: 26/Apr/21 11:24

Description

Array types should have an Iterable trait that generates plain or nullable iterators.

pub trait Iterable<'a>
where Self::IterType: std::iter::Iterator
{
    type IterType;

    fn iter(&'a self) -> Self::IterType;
    fn iter_nulls(&'a self) -> NullableIterator<Self::IterType>;
}

IterType depends on the array type from standard slice iterators for primitive types, string iterators for UTF8 types and composite iterators (generating other iterators) for list, struct and dictionary types.

The NullableIterator type should bundle a null bitmap pointer with another iterator type to form a composite iterator that returns an option:

/// Convert any iterator to a nullable iterator by using the null bitmap.
#[derive(Debug, PartialEq, Clone)]
pub struct NullableIterator<T: Iterator> {
    iter: T,
    i: usize,
    null_bitmap: *const u8,
}

impl<T: Iterator> NullableIterator<T> {
    fn from(iter: T, null_bitmap: &Option<Bitmap>, offset: usize) -> Self;
}

For more details, some exploratory work has been done here: https://github.com/andy-thomason/arrow/blob/ARROW-iterators/rust/arrow/src/array/array.rs#L1711