Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-6779

[Python] Conversion from datetime.datetime to timstamp('ns') can overflow

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Duplicate
    • None
    • None
    • Python
    • None

    Description

      In the python conversion of datetime scalars, there is no check for integer overflow:

      In [32]: pa.array([datetime.datetime(3000, 1, 1)], pa.timestamp('ns'))                                                                                                                                             
      Out[32]: 
      <pyarrow.lib.TimestampArray object at 0x7f5bddec00a8>
      [
        1830-11-23 00:50:52.580896768
      ]
      

      So in case the target type has nanosecond unit, this can give wrong results (I don't think the other resolutions can reach overflow, given the limited range of years of datetime.datetime).

      We should probably check for this case and raise an error.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              jorisvandenbossche Joris Van den Bossche
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Slack

                  Issue deployment