[ARROW-15898] [CI][Packaging] Anaconda storage size exceeded

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 8.0.0
• Component/s: Continuous Integration, Packaging
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/31328
• Assignee: Antoine Pitrou
• Reporter: Antoine Pitrou
• Created: 10/Mar/22 11:22
• Updated: 11/Jan/23 11:40
• Resolved: 15/Mar/22 09:27

Description

Our Anaconda storage size for nightlies is exceeded:

Uploading file "arrow-nightlies/arrow-cpp-proc/8.0.0.dev201/osx-64/arrow-cpp-proc-8.0.0.dev201-cpu.tar.bz2"
[WARNING] Distribution "osx-64/arrow-cpp-proc-8.0.0.dev201-cpu.tar.bz2" already exists. Removing.
[ERROR] ('Storage requirements exceeded (3221225472 bytes). Payment is required to add a file. Please go to https://anaconda.org/binstar.settings/billing to update your plan', 402)
##[error]Bash exited with code '1'.

(from https://dev.azure.com/ursacomputing/crossbow/_build/results?buildId=21174&view=logs&j=cf796865-97b7-5cd1-be8e-6e00ce4fd8cf&t=9033228b-6263-5857-d694-7e9a2323817b )

The conda-clean job is supposed to remove old versions but it actually removes only a subset of them:
https://dev.azure.com/ursacomputing/crossbow/_build/results?buildId=21142&view=logs&j=0da5d1d9-276d-5173-c4c4-9d4d4ed14fdb&t=d9b15392-e4ce-5e4c-0c8c-b69645229181&l=98
https://anaconda.org/arrow-nightlies/repo/files

Issue Links

• links to: GitHub Pull Request #12604