Details
-
Bug
-
Status: Open
-
Minor
-
Resolution: Unresolved
-
3.4.3
-
None
-
None
Description
The `concat` method has a strict type check, that raises a misleading error:
Note that the type raised is of `objs`, rather than `obj`, so a list of various objects will say that it cannot concatenate objects of type list, rather than the failed internal types.
Additionally, this strictly checks for pandas-on-spark Series and DataFrames; since both objects will happily convert a naive Pandas object, something like
objs = [DataFrame(x) if isinstance(x, pd.Dataframe) else Series(x) if isinstance(x, pd.Series) else x for x in objs]
would trivially make this work in those cases and prevent a different strange error reporting that a dataframe wasn't valid in a dataframe concatenation.