[BEAM-10004] ZeroDivisionError if source bundle smaller than 1mb

• Type: Bug
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Affects Version/s: 2.20.0
• Fix Version/s: 2.23.0
• Component/s: io-py-mongodb
• Assignee: Corvin Deboeser
• Reporter: Corvin Deboeser
• Created: 15/May/20 07:21
• Updated: 07/Jul/20 02:51
• Resolved: 07/Jul/20 02:51

Description

If the desired_bundle_size is lower than 1mb, then split returns only SourceBundles with weight=0 which leads to a ZeroDivisionError down the line. 

ZeroDivisionError: float division by zero

This error is raised from _compute_cumulative_weights here:

https://github.com/apache/beam/blob/9f0cb649d39ee6236ea27f111acb4b66591a80ec/sdks/python/apache_beam/io/concat_source.py#L154

 

Worked for me: Pulling the truncation from _get_split_keys (here) into split instead.

 

 

Issue Links

• links to: GitHub Pull Request #12142