[ARROW-15233] pyarrow.dataset.dataset loses type information when reading parquet files

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 6.0.1
• Component/s: Python
• Environment: Ubuntu 20.2, Python 3.8.10
• External issue URL: https://github.com/apache/arrow/issues/30731
• Assignee: Unassigned
• Reporter: Jim Fulton
• Created: 01/Jan/22 00:48
• Updated: 11/Jan/23 08:45
• Resolved: 01/Jan/22 19:29

Description

When reading a parquet containing time data:

 

>>> import pyarrow.dataset
>>> ds = pyarrow.dataset.dataset('foo.parquet', format='parquet')
>>> ds.schema[1].type
DataType(time32[ms])

I get DataType rather than Time32Type, which means I can't query time units.

I assume this is an issue for other non-basic types.

I can write code to scrape the type's string representation. Is there a better way?