Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-4090

[Python] Table.flatten() doesn't work recursively

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Python

      Description

      It seems that the pyarrow.Table.flatten() function is not working recursively nor providing a parameter to do it.

      test1c_data = {'level1-A': 'abc',
                     'level1-B': 112233,
                     'level1-C': {'x': 123.111, 'y': 123.222, 'z': 123.333}
                    }
      test1c_type = pa.struct([('level1-A', pa.string()),
                               ('level1-B', pa.int32()),
                               ('level1-C', pa.struct([('x', pa.float64()),
                                                       ('y', pa.float64()),
                                                       ('z', pa.float64())
                                                      ]))
                              ])
      test1c_array = pa.array([test1c_data]*5, type=test1c_type)
      test1c_table = pa.Table.from_arrays([test1c_array], names=['msg']) 
      
      print('{}\n\n{}\n\n{}'.format(test1c_table.schema,
                                    test1c_table.flatten().schema,
                                    test1c_table.flatten().flatten().schema))
      

      output:

      msg: struct<level1-A: string, level1-B: int32, level1-C: struct<x: double, y: double, z: double>>
      child 0, level1-A: string
      child 1, level1-B: int32
      child 2, level1-C: struct<x: double, y: double, z: double>
      child 0, x: double
      child 1, y: double
      child 2, z: double

      msg.level1-A: string
      msg.level1-B: int32
      msg.level1-C: struct<x: double, y: double, z: double>
      child 0, x: double
      child 1, y: double
      child 2, z: double

      msg.level1-A: string
      msg.level1-B: int32
      msg.level1-C.x: double
      msg.level1-C.y: double
      msg.level1-C.z: double

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              FJ_Sanchez Francisco Sanchez
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 10m
                10m