[ARROW-6854] [Dataset][C++] RecordBatchProjector is not thread safe

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: C++
• External issue URL: https://github.com/apache/arrow/issues/23184
• Assignee: Unassigned
• Reporter: Francois Saint-Jacques
• Created: 11/Oct/19 13:14
• Updated: 11/Jan/23 07:49

Description

While working on ARROW-6769 I noted that RecordbBatchProjector is not thread safe. My goal is to use this class to wrap the ScanTaskIterator in another ScanTaskIterator that projects, so producer (fragments) don't have to know about this schema. The issue is that ScanTask are expected to run on concurrent thread. The projector will be invoked by multiple thread.

The lack of concurrency safety is due to adaptivity of input schemas and `SetInputSchema` stores in a local cache. I suggest we refactor into 2 classes. 

1. `RecordBatchProjector` which will work with a static `from` schema, i.e. no adaptivity. The schema is defined at construct time. This class is thread safe to invoke after construction since no local modification is done.
2. `AdaptiveRecordBatchProjector` which will have a cache map[schema_hash, std::shared_ptr<RecordBatchProjector>] protected with a mutex.