[ARROW-10888] [Rust] Tracking issue for safety issues

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Invalid
• Component/s: Rust
• External issue URL: https://github.com/apache/arrow/issues/26822
• Assignee: Unassigned
• Reporter: Andy Grove
• Created: 12/Dec/20 18:58
• Updated: 11/Jan/23 08:16
• Resolved: 26/Apr/21 13:00

Description

We have a number of functions in the Rust code base that use unsafe code but they are not declared as unsafe or documented as to why we are guaranteeing that they are actually safe.

I have seen recent criticism of the project on social media because of this and it is something that we should address.

If anyone is interested in working on this, please create sub-tasks under this JIRA.

Here is a count by source file as of 19 Dec 2020 of the work "unsafe".

./arrow-pyarrow-integration-testing/src/lib.rs:3
./parquet/src/util/hash_util.rs:6
./parquet/src/util/bit_packing.rs:34
./parquet/src/util/bit_util.rs:1
./parquet/src/data_type.rs:12
./parquet/src/arrow/record_reader.rs:5
./parquet/src/arrow/array_reader.rs:8
./arrow/src/buffer.rs:32
./arrow/src/compute/kernels/comparison.rs:2
./arrow/src/compute/kernels/substring.rs:1
./arrow/src/compute/kernels/arithmetic.rs:1
./arrow/src/compute/kernels/length.rs:2
./arrow/src/compute/util.rs:1
./arrow/src/util/bit_chunk_iterator.rs:6
./arrow/src/util/bit_util.rs:11
./arrow/src/util/integration_util.rs:1
./arrow/src/bytes.rs:5
./arrow/src/arch/avx512.rs:4
./arrow/src/bitmap.rs:1
./arrow/src/datatypes.rs:12
./arrow/src/zz_memory_check.rs:1
./arrow/src/ffi.rs:18
./arrow/src/array/transform/mod.rs:1
./arrow/src/array/array.rs:1
./arrow/src/array/ffi.rs:2
./arrow/src/array/array_list.rs:4
./arrow/src/array/array_string.rs:2
./arrow/src/array/builder.rs:6
./arrow/src/array/array_boolean.rs:4
./arrow/src/array/raw_pointer.rs:2
./arrow/src/array/data.rs:1
./arrow/src/array/array_primitive.rs:4
./arrow/src/array/array_binary.rs:4
./arrow/src/array/array_union.rs:2
./arrow/src/memory.rs:10
./arrow/src/ipc/gen/File.rs:2
./arrow/src/ipc/gen/Message.rs:2
./arrow/src/ipc/gen/Schema.rs:2
 

 

Sub-Tasks

• [Rust] Document our approach to unsafe code in README: Resolved
• [Rust] Document use of unsafe in compute kernels: Closed
• [Rust] Document use of unsafe in memory.rs: Closed
• [Rust] Document use of unsafe in IPC: Closed
• [Rust] Document use of unsafe in buffer.rs: Closed
• [Rust] Document use of unsafe in builder.rs: Closed
• [Rust] Document use of unsafe in utils: Closed
• [Rust] Document use of unsafe in src/array/*.rs: Closed
• [Rust] Document use of unsafe in remaining files: Closed
• [Rust] Document use of unsafe in ffi.rs: Closed
• [Rust] Document use of unsafe in datatypes.rs: Closed
• [Rust] Document use of unsafe in parquet crate: Closed
• [Rust] Update unsafe guidelines for adding JIRA references: Resolved