[ARROW-15026] [Python] datetime.timedelta to pyarrow.duration('us') silently overflows

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 11.0.0
• Component/s: Python
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/30544
• Assignee: Anja Boskovic
• Reporter: Andreas Rappold
• Created: 08/Dec/21 13:51
• Updated: 11/Jan/23 08:43
• Resolved: 15/Nov/22 18:19

Description

 

Hi! This reproduces the issue:

# python 3.9.9
# pyarrow 6.0.1
import datetime
import pyarrow
d = datetime.timedelta(days=-106751992, seconds=71945, microseconds=224192)
pyarrow.scalar(d)
# <pyarrow.DurationScalar: datetime.timedelta(days=-106751992, seconds=71945, microseconds=224192)>
pyarrow.scalar(d).as_py() == d
# True
d2 = d - datetime.timedelta(microseconds=1)
pyarrow.scalar(d2)
# <pyarrow.DurationScalar: datetime.timedelta(days=106751991, seconds=14454, microseconds=775807)>
pyarrow.scalar(d2).as_py() == d2
# False

Other conversions (e.g. to int*) raise an exception instead. I didn't check if duration overflows for too large timedeltas. If its easy to fix, point me in the right direction and I try to create a PR. Thanks

 

Issue Links

• links to: GitHub Pull Request #13718