Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-14004

[Python] to_pandas() converts to float instead of using pandas nullable types

    XMLWordPrintableJSON

Details

    Description

      We've noticed that when converting an Arrow Table to pandas using `.to_pandas()` integer columns with null values get converted to float instead of using pandas nullable types.

      If the column was created with pandas first it is correctly preserved (I guess it's using stored metadata for this).

      I've attached a screenshot showing this behavior.

      As currently there is support for nullable types in pandas, just as in Arrow, it would be great to use these types when dealing with columns with null values.

      If you are reticent to change this behavior, a param would be nice too (e.g. `to_pandas(use_nullable_types: True)`).

       

      Attachments

        1. image.png
          36 kB
          Miguel Cantón Cortés

        Issue Links

          Activity

            People

              jorisvandenbossche Joris Van den Bossche
              miwelc Miguel Cantón Cortés
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 20m
                  1h 20m