[BEAM-12541] fillna(value=DataFrame/Series) does not correctly fill values

• Type: Bug
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Fix Version/s: 2.32.0
• Component/s: dsl-dataframe
• Assignee: Brian Hulette
• Reporter: Brian Hulette
• Created: 25/Jun/21 20:17
• Updated: 29/Jun/21 13:44
• Resolved: 28/Jun/21 19:54

Description

A new doctest was added in 1.3.0 that verifies fillna with a DataFrame value:

df = pd.DataFrame([[np.nan, 2, np.nan, 0],
                   [3, 4, np.nan, 1],
                   [np.nan, np.nan, np.nan, 5],
                   [np.nan, 3, np.nan, 4]],
                  columns=list("ABCD"))
df2 = pd.DataFrame(np.zeros((4, 4)), columns=list("ABCE"))
df.fillna(df2)
    A   B   C   D
0   0.0 2.0 0.0 0
1   3.0 4.0 0.0 1
2   0.0 0.0 0.0 5
3   0.0 3.0 0.0 4

Unfortunately, we fail this test. The root cause is that we don't require Index partitioning when fillna is called with a DataFrame/Series value.

Issue Links

• links to: GitHub Pull Request #15086