[ARROW-2447] [C++] Create a device abstraction

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.9.0
• Fix Version/s: 0.17.0
• Component/s: C++, GPU
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/18584
• Assignee: Antoine Pitrou
• Reporter: Antoine Pitrou
• Created: 11/Apr/18 11:14
• Updated: 11/Jan/23 07:21
• Resolved: 12/Feb/20 23:28

Description

Right now, a plain Buffer doesn't carry information about where it actually lies. That information also cannot be passed around, so you get APIs like PlasmaClient which take or return device number integers, and have implementations which hardcode operations on CUDA buffers. Also, unsuspecting receivers of a Buffer pointer may try to act on the underlying memory without knowing whether it's CPU-reachable or not.

Here is a sketch for a proposed Device abstraction:

class Device {
    enum DeviceKind { KIND_CPU, KIND_CUDA };

    virtual DeviceKind kind() const;
    //MemoryPool* default_memory_pool() const;
    //std::shared_ptr<Buffer> Allocate(...);
};

class CpuDevice : public Device {};

class CudaDevice : public Device {
    int device_num() const;
};

class Buffer {
    virtual DeviceKind device_kind() const;
    virtual std::shared_ptr<Device> device() const;
    virtual bool on_cpu() const {
        return true;
    }

    const uint8_t* cpu_data() const {
        return on_cpu() ? data() : nullptr;
    }
    uint8_t* cpu_mutable_data() {
        return on_cpu() ? mutable_data() : nullptr;
    }

    virtual CopyToCpu(std::shared_ptr<Buffer> dest) const;
    virtual CopyFromCpu(std::shared_ptr<Buffer> src);
};

class CudaBuffer : public Buffer {
    virtual bool on_cpu() const {
        return false;
    }
};

CopyBuffer(std::shared_ptr<Buffer> dest, const std::shared_ptr<Buffer> src);

Issue Links

• is depended upon by: ARROW-7748 [C++] [Cuda] Cache CUDA contexts
• is depended upon by: ARROW-1824 [C++] Add better GPU support for RecordBatch objects in arrow::ipc::*
• is part of: ARROW-1055 [C++] GPU support library development
• relates to: ARROW-2446 [C++] SliceBuffer on CudaBuffer should return CudaBuffer
• supercedes: ARROW-1470 [C++] Add BufferAllocator abstract interface
• links to: GitHub Pull Request #6295