When running a Spark job I am packaging all Python dependencies in a zip-file which is distributed to the nodes. I am using AWS Glue to execute the Spark job.
One of the dependencies included is the python package avro-python3. However, this cannot be used since VERSION.txt cannot be opened when the package is included as a zip. It errors out in _init_.py when trying to open the file. The file VERSION.txt is present, however the path given to open() looks something like "/home/user/custom-site-package.zip/avro/VERSION.txt".
On a side note, the package has zip_safe set to true, so I expect this to work
I have opened a PR on Github with a possible solution.