[BEAM-11266] Cannot use Python MongoDB connector with Atlas MongoDB

• Type: Bug
• Status: Triage Needed
• Priority: P2
• Resolution: Fixed
• Affects Version/s: 2.25.0
• Fix Version/s: 2.27.0
• Component/s: io-py-mongodb
• Labels: mongodb python
• Environment: Google Cloud Dataflow
• Assignee: Yichi Zhang
• Reporter: Eugene Nikolaiev
• Created: 15/Nov/20 12:11
• Updated: 13/Apr/23 11:03
• Resolved: 06/Dec/20 16:30

Description

Cannot use the Python MongoDB connector with a managed Atlas instance. The current implementations makes use of splitVector which is a high-privilege function that cannot be assigned to any user in Atlas. Getting error:

pymongo.errors.OperationFailure: not authorized on properties to execute command
 { splitVector: "properties.properties", keyPattern: { _id: 1 },
...

BEAM-4567 addressed the same issue in Java connector.

Proposed solution for Python is to add bucket_auto option for the connector which would configure it to use @bucketAuto MongoDB aggregation instead of splitVector command:

pipeline | ReadFromMongoDB(uri='mongodb+srv://user:[e-mail]',
                           db='testdb',
                           coll='input',
                           bucket_auto=True)

Issue Links

• is related to: BEAM-4567 Can't use mongo connector with Atlas MongoDB
• links to: GitHub Pull Request #13350