[ARROW-11394] [Rust] Slice + Concat incorrect for structs

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.0.1, 4.0.0
• Component/s: Rust
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/27286
• Assignee: Ben Chambers
• Reporter: Ben Chambers
• Created: 27/Jan/21 04:57
• Updated: 11/Jan/23 08:19
• Resolved: 31/Jan/21 14:37

Description

If you slice an array and then use it with concat you get different behaviors when using a primitive array (Float64 in the examples) or a struct array.

In the case of a float, the result is what I'd expect – it concatenates the elements from the slice.

In the case of a struct, it is a bit surprising – the result has the length of the slice, but starts at the beginning of the original array.

// #[test]
fn test_repro() {
    // Create float and struct array.
    let float_array: ArrayRef = Arc::new(Float64Array::from(vec![1.0, 2.0, 3.0, 4.0]));
    let struct_array = Arc::new(StructArray::from(vec![(
        Field::new("field", DataType::Float64, true),
        float_array.clone(),
    )]));

    // Slice the float array and verify result is [3.0, 4.0]
    let float_array_slice_ref = float_array.slice(2, 2);
    let float_array_slice = float_array_slice_ref
        .as_any()
        .downcast_ref::<PrimitiveArray<Float64Type>>()
        .unwrap();
    assert_eq!(float_array_slice, &Float64Array::from(vec![3.0, 4.0]));

    // Slice the struct array and verify result is [3.0, 4.0]
    let struct_array_slice_ref = struct_array.slice(2, 2);
    let struct_array_slice = struct_array_slice_ref
        .as_any()
        .downcast_ref::<StructArray>()
        .unwrap();
    let struct_array_slice_floats = struct_array_slice
        .column(0)
        .as_any()
        .downcast_ref::<PrimitiveArray<Float64Type>>()
        .unwrap();
    assert_eq!(
        struct_array_slice_floats,
        &Float64Array::from(vec![3.0, 4.0])
    );

    // Concat the float array, and verify the result is still [3.0, 4.0].
    let concat_float_array_ref =
        arrow::compute::kernels::concat::concat(&[float_array_slice]).unwrap();
    let concat_float_array = concat_float_array_ref
        .as_any()
        .downcast_ref::<PrimitiveArray<Float64Type>>()
        .unwrap();
    assert_eq!(concat_float_array, &Float64Array::from(vec![3.0, 4.0]));

    // Concat the struct array and expect it to match the float array [3.0, 4.0].
    let concat_struct_array_ref =
        arrow::compute::kernels::concat::concat(&[struct_array_slice]).unwrap();
    let concat_struct_array = concat_struct_array_ref
        .as_any()
        .downcast_ref::<StructArray>()
        .unwrap();
    let concat_struct_array_floats = concat_struct_array
        .column(0)
        .as_any()
        .downcast_ref::<PrimitiveArray<Float64Type>>()
        .unwrap();
    // This is what is actually returned
    assert_eq!(
        concat_struct_array_floats,
        &Float64Array::from(vec![1.0, 2.0])
    );

    // This is what I'd expect, but fails:
    assert_eq!(
        concat_struct_array_floats,
        &Float64Array::from(vec![3.0, 4.0])
    );
}

Issue Links

• links to: GitHub Pull Request #9339