[BEAM-10626] tests fails on windows - pandas_doctests_test.DoctestTest

• Type: Bug
• Status: Open
• Priority: P3
• Resolution: Unresolved
• Component/s: sdk-py-core, testing
• Assignee: Unassigned
• Reporter: Tobiasz Kedzierski
• Created: 03/Aug/20 11:20
• Updated: 04/Jun/22 17:09

Description

apache_beam.dataframe.pandas_doctests_test.DoctestTest fails on windows

Link to the github workflow run with mentioned error:
https://github.com/TobKed/beam/runs/939434818?check_suite_focus=true
partial log:

2020-08-03T07:20:24.2575042Z ================================== FAILURES ===================================
2020-08-03T07:20:24.2687843Z ______________________ DoctestTest.test_dataframe_tests _______________________
2020-08-03T07:20:24.2688268Z [gw0] win32 – Python 3.6.8 d:\a\beam\beam\sdks\python\target\.tox\py36-win\scripts\python.exe
2020-08-03T07:20:24.2688555Z
2020-08-03T07:20:24.2689341Z self = <apache_beam.dataframe.pandas_doctests_test.DoctestTest testMethod=test_dataframe_tests>
2020-08-03T07:20:24.2689548Z
2020-08-03T07:20:24.2689775Z def test_dataframe_tests(self):
2020-08-03T07:20:24.2690019Z result = doctests.testmod(
2020-08-03T07:20:24.2690221Z pd.core.frame,
2020-08-03T07:20:24.2690442Z use_beam=False,
2020-08-03T07:20:24.2690631Z skip={
2020-08-03T07:20:24.2690860Z 'pandas.core.frame.DataFrame.T': ['*'],
2020-08-03T07:20:24.2691084Z 'pandas.core.frame.DataFrame.agg': ['*'],
2020-08-03T07:20:24.2691349Z 'pandas.core.frame.DataFrame.aggregate': ['*'],
2020-08-03T07:20:24.2691579Z 'pandas.core.frame.DataFrame.append': ['*'],
2020-08-03T07:20:24.2691806Z 'pandas.core.frame.DataFrame.apply': ['*'],
2020-08-03T07:20:24.2692034Z 'pandas.core.frame.DataFrame.applymap': ['df ** 2'],
2020-08-03T07:20:24.2692260Z 'pandas.core.frame.DataFrame.assign': ['*'],
2020-08-03T07:20:24.2692485Z 'pandas.core.frame.DataFrame.axes': ['*'],
2020-08-03T07:20:24.2692970Z 'pandas.core.frame.DataFrame.combine': ['*'],
2020-08-03T07:20:24.2693371Z 'pandas.core.frame.DataFrame.combine_first': ['*'],
2020-08-03T07:20:24.2693631Z 'pandas.core.frame.DataFrame.corr': ['*'],
2020-08-03T07:20:24.2693889Z 'pandas.core.frame.DataFrame.count': ['*'],
2020-08-03T07:20:24.2694287Z 'pandas.core.frame.DataFrame.cov': ['*'],
2020-08-03T07:20:24.2694663Z 'pandas.core.frame.DataFrame.dot': ['*'],
2020-08-03T07:20:24.2694920Z 'pandas.core.frame.DataFrame.drop': ['*'],
2020-08-03T07:20:24.2695215Z 'pandas.core.frame.DataFrame.eval': ['*'],
2020-08-03T07:20:24.2695549Z 'pandas.core.frame.DataFrame.explode': ['*'],
2020-08-03T07:20:24.2695887Z 'pandas.core.frame.DataFrame.fillna': ['*'],
2020-08-03T07:20:24.2696218Z 'pandas.core.frame.DataFrame.info': ['*'],
2020-08-03T07:20:24.2696594Z 'pandas.core.frame.DataFrame.isin': ['*'],
2020-08-03T07:20:24.2697044Z 'pandas.core.frame.DataFrame.iterrows': ["print(df['int'].dtype)"],
2020-08-03T07:20:24.2697476Z 'pandas.core.frame.DataFrame.join': ['*'],
2020-08-03T07:20:24.2697743Z 'pandas.core.frame.DataFrame.melt': ['*'],
2020-08-03T07:20:24.2697999Z 'pandas.core.frame.DataFrame.memory_usage': ['*'],
2020-08-03T07:20:24.2698270Z 'pandas.core.frame.DataFrame.merge': ['*'],
2020-08-03T07:20:24.2698697Z # Not equal to df.agg('mode', axis='columns', numeric_only=True)
2020-08-03T07:20:24.2698918Z 'pandas.core.frame.DataFrame.mode': [
2020-08-03T07:20:24.2699338Z "df.mode(axis='columns', numeric_only=True)"
2020-08-03T07:20:24.2699582Z ],
2020-08-03T07:20:24.2699832Z 'pandas.core.frame.DataFrame.nlargest': ['*'],
2020-08-03T07:20:24.2700172Z 'pandas.core.frame.DataFrame.nsmallest': ['*'],
2020-08-03T07:20:24.2700434Z 'pandas.core.frame.DataFrame.nunique': ['*'],
2020-08-03T07:20:24.2700789Z 'pandas.core.frame.DataFrame.pivot': ['*'],
2020-08-03T07:20:24.2701145Z 'pandas.core.frame.DataFrame.pivot_table': ['*'],
2020-08-03T07:20:24.2701476Z 'pandas.core.frame.DataFrame.query': ['*'],
2020-08-03T07:20:24.2701731Z 'pandas.core.frame.DataFrame.reindex': ['*'],
2020-08-03T07:20:24.2702072Z 'pandas.core.frame.DataFrame.reindex_axis': ['*'],
2020-08-03T07:20:24.2702429Z 'pandas.core.frame.DataFrame.rename': ['*'],
2020-08-03T07:20:24.2702861Z # Raises right exception, but testing framework has matching issues.
2020-08-03T07:20:24.2703235Z 'pandas.core.frame.DataFrame.replace': [
2020-08-03T07:20:24.2703673Z "df.replace(

{'a string': 'new value', True: False}

) # raises"
2020-08-03T07:20:24.2704039Z ],
2020-08-03T07:20:24.2704317Z # Uses unseeded np.random.
2020-08-03T07:20:24.2704544Z 'pandas.core.frame.DataFrame.round': ['*'],
2020-08-03T07:20:24.2704780Z 'pandas.core.frame.DataFrame.set_index': ['*'],
2020-08-03T07:20:24.2705005Z 'pandas.core.frame.DataFrame.transpose': [
2020-08-03T07:20:24.2705277Z 'df1_transposed.dtypes', 'df2_transposed.dtypes'
2020-08-03T07:20:24.2705496Z ],
2020-08-03T07:20:24.2705715Z 'pandas.core.frame.DataFrame.to_sparse': ['type(df)'],
2020-08-03T07:20:24.2705938Z # Uses df.index
2020-08-03T07:20:24.2706204Z 'pandas.core.frame.DataFrame.to_records': ['*'],
2020-08-03T07:20:24.2706415Z })
2020-08-03T07:20:24.2706626Z > self.assertEqual(result.failed, 0)
2020-08-03T07:20:24.2706839Z E AssertionError: 1 != 0
2020-08-03T07:20:24.2706961Z
2020-08-03T07:20:24.2707176Z apache_beam\dataframe\pandas_doctests_test.py:88: AssertionError
2020-08-03T07:20:24.2707835Z ---------------------------- Captured stdout call -----------------------------
2020-08-03T07:20:24.2708148Z **********************************************************************
2020-08-03T07:20:24.2710505Z File "d:\a\beam\beam\sdks\python\target\.tox\py36-win\lib\site-packages\pandas\core\frame.py", line 3387, in pandas.core.frame.DataFrame.select_dtypes
2020-08-03T07:20:24.2710919Z Failed example:
2020-08-03T07:20:24.2711135Z df.select_dtypes(exclude=['int'])
2020-08-03T07:20:24.2711328Z Expected:
2020-08-03T07:20:24.2711519Z b c
2020-08-03T07:20:24.2711708Z 0 True 1.0
2020-08-03T07:20:24.2711891Z 1 False 2.0
2020-08-03T07:20:24.2712072Z 2 True 1.0
2020-08-03T07:20:24.2712253Z 3 False 2.0
2020-08-03T07:20:24.2712435Z 4 True 1.0
2020-08-03T07:20:24.2712621Z 5 False 2.0
2020-08-03T07:20:24.2712804Z Got:
2020-08-03T07:20:24.2712995Z a b c
2020-08-03T07:20:24.2713186Z 0 1 True 1.0
2020-08-03T07:20:24.2713374Z 1 2 False 2.0
2020-08-03T07:20:24.2713564Z 2 1 True 1.0
2020-08-03T07:20:24.2713716Z 3 2 False 2.0
2020-08-03T07:20:24.2714008Z 4 1 True 1.0
2020-08-03T07:20:24.2714283Z 5 2 False 2.0
2020-08-03T07:20:24.2714492Z **********************************************************************
2020-08-03T07:20:24.2714688Z 1 items had failures:
2020-08-03T07:20:24.2714891Z 1 of 5 in pandas.core.frame.DataFrame.select_dtypes
2020-08-03T07:20:24.2715088Z **Test Failed** 1 failures.

Issue Links

• relates to: BEAM-10623 Add workflow to run Beam python tests on Linux/Windows/Mac platforms
• relates to: BEAM-9496 Add a Dataframe API for Python