At no time in the Arrow codebase do we validate that the dictionary indices are in bounds. It seems that pandas is overly trusting of the validity of the indices. So we should add a method someplace to validate that the dictionary non-null indices are not out of bounds (perhaps in CategoricalBlock::WriteIndices).
As an aside: there may be other times when doing analytics on categorical data that external data will have out of bounds index values. We should plan for these and decide whether to raise an exception or treat them as null