Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-5295

[Python] accept pyarrow values / scalars in constructor functions ?

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Python
    • Labels:
      None

      Description

      Currently, functions like pyarrow.array don't accept pyarrow Arrays, or also not scalars of it:

      In [42]: arr = pa.array([1, 2, 3])
      
      In [43]: pa.array(arr)
      ...
      ArrowInvalid: Could not convert 1 with type pyarrow.lib.Int64Value: did not recognize Python value type when inferring an Arrow data type
      
      In [44]: pa.array(list(arr))
      ...
      ArrowInvalid: Could not convert 1 with type pyarrow.lib.Int64Value: did not recognize Python value type when inferring an Arrow data type
      

      Do we want to allow those / recognize those here? (the first case could even have a fastpath, as we don't need to do it element by element).

      Also scalars are not supported:

      In [46]: type(arr.sum())
      Out[46]: pyarrow.lib.Int64Scalar
      
      In [47]: pa.array([arr.sum()])
      ...
      ArrowInvalid: Could not convert 6 with type pyarrow.lib.Int64Scalar: did not recognize Python value type when inferring an Arrow data type
      

      And also in other functions we don't accept arrow scalars / values:

      In [48]: string = pa.array(['a'])[0]
      
      In [49]: type(string)
      Out[49]: pyarrow.lib.StringValue
      
      In [50]: pa.field(string, pa.int64())
      ...
      TypeError: expected bytes, pyarrow.lib.StringValue found
      

       

       

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              jorisvandenbossche Joris Van den Bossche
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: