[BEAM-10624] tests fails on windows - pyarrow stores int always as 32 bit

• Type: Bug
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Fix Version/s: 2.25.0
• Component/s: sdk-py-core, testing
• Assignee: Brian Hulette
• Reporter: Tobiasz Kedzierski
• Created: 03/Aug/20 10:53
• Updated: 15/May/21 17:58
• Resolved: 15/May/21 17:58

Description

Failing test:
apache_beam.testing.datatype_inference_test.DatatypeInferenceTest.test_infer_pyarrow_schema

Link to the github workflow run with mentioned error:
https://github.com/TobKed/beam/runs/937414511?check_suite_focus=true

partial log:
2020-08-02T11:59:00.9369328Z ================================== FAILURES ===================================
2020-08-02T11:59:00.9470134Z ___________ DatatypeInferenceTest.test_infer_pyarrow_schema_1_main ____________
2020-08-02T11:59:00.9470741Z [gw4] win32 – Python 3.7.8 d:\a\beam\beam\sdks\python\target\.tox\py37-win\scripts\python.exe
2020-08-02T11:59:00.9471799Z
2020-08-02T11:59:00.9472229Z a = (<apache_beam.testing.datatype_inference_test.DatatypeInferenceTest testMethod=test_infer_pyarrow_schema_1_main>,)
2020-08-02T11:59:00.9472480Z
2020-08-02T11:59:00.9472784Z @wraps(func)
2020-08-02T11:59:00.9473094Z def standalone_func(*a):
2020-08-02T11:59:00.9473370Z > return func(*(a + p.args), **p.kwargs)
2020-08-02T11:59:00.9473579Z
2020-08-02T11:59:00.9473858Z target\.tox\py37-win\lib\site-packages\parameterized\parameterized.py:530:
2020-08-02T11:59:00.9474175Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
2020-08-02T11:59:00.9474571Z apache_beam\testing\datatype_inference_test.py:183: in test_infer_pyarrow_schema
2020-08-02T11:59:00.9474893Z self.assertEqual(pyarrow_schema, schema)
2020-08-02T11:59:00.9475190Z E AssertionError: a: in[14 chars]c: string
2020-08-02T11:59:00.9475478Z E d: list<item: int32>
2020-08-02T11:59:00.9475766Z E child 0, item: int32
2020-08-02T11:59:00.9476074Z E e: binary != a: in[14 chars]c: string
2020-08-02T11:59:00.9476366Z E d: list<item: int64>
2020-08-02T11:59:00.9476668Z E child 0, item: int64
2020-08-02T11:59:00.9476948Z E e: binary
2020-08-02T11:59:00.9477336Z ______ DatatypeInferenceTest.test_infer_pyarrow_schema_2_main_nullified _______
2020-08-02T11:59:00.9477924Z [gw4] win32 – Python 3.7.8 d:\a\beam\beam\sdks\python\target\.tox\py37-win\scripts\python.exe
2020-08-02T11:59:00.9478180Z
2020-08-02T11:59:00.9478977Z a = (<apache_beam.testing.datatype_inference_test.DatatypeInferenceTest testMethod=test_infer_pyarrow_schema_2_main_nullified>,)
2020-08-02T11:59:00.9479255Z
2020-08-02T11:59:00.9479512Z @wraps(func)
2020-08-02T11:59:00.9479809Z def standalone_func(*a):
2020-08-02T11:59:00.9480081Z > return func(*(a + p.args), **p.kwargs)
2020-08-02T11:59:00.9480288Z
2020-08-02T11:59:00.9480562Z target\.tox\py37-win\lib\site-packages\parameterized\parameterized.py:530:
2020-08-02T11:59:00.9481023Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
2020-08-02T11:59:00.9481594Z apache_beam\testing\datatype_inference_test.py:183: in test_infer_pyarrow_schema
2020-08-02T11:59:00.9481922Z self.assertEqual(pyarrow_schema, schema)
2020-08-02T11:59:00.9482390Z E AssertionError: a: in[14 chars]c: string
2020-08-02T11:59:00.9482697Z E d: list<item: int32>
2020-08-02T11:59:00.9482956Z E child 0, item: int32
2020-08-02T11:59:00.9483261Z E e: binary != a: in[14 chars]c: string
2020-08-02T11:59:00.9483524Z E d: list<item: int64>
2020-08-02T11:59:00.9483812Z E child 0, item: int64
2020-08-02T11:59:00.9484071Z E e: binary

Issue Links

• relates to: BEAM-10623 Add workflow to run Beam python tests on Linux/Windows/Mac platforms