Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.5.0
    • Component/s: Python
    • Labels:
      None

      Description

      Hi there,

      I am trying to use pyarrow to convert CSV files to Parquet for use with Redshift Spectrum. I've got everything sorted... almost

      Unfortunately, the only format they accept for timestamp columns is int96. I understand that int96 timestamps are unofficial/deprecated, but unfortunately it's what I am stuck with for integrating with, at least for the moment. I contacted Amazon support and 64-bit timestamp support has been added to the list of feature requests, but it's unclear when it will be prioritized/added/released.

      In the meantime, I am thinking of adding write support for int96 columns to arrow. Would that be a welcome addition?

      Thanks,
      Colin

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                xhochy Uwe L. Korn
                Reporter:
                cnichols colin nichols
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: