PyPI Cache

August 2, 2022, Joachim Folz

We’ve installed a local cache for PyPI packages. Hosting frequently installed packages locally is nicer to PyPI and should significantly speed up package installation. Read below how to use it.

How to use #

You have two options to point pip at our cache:

1. Command line switches

   pip install --no-cache --index-url http://pypi-cache/index --trusted-host pypi-cache <package>
   

2. Environment variables

   export PIP_INDEX_URL=http://pypi-cache/index
   export PIP_TRUSTED_HOST=pypi-cache
   export PIP_NO_CACHE=true
   pip install <package>
   

Second option can also work nicely with srun --export. We add --no-cache / PIP_NO_CACHE=true since it doesn’t make sense to cache packages locally when they’re available from a fast cache.

Please report problems #

This is an early test, so please report any problems you experience. If there’s none this will become the default for all cluster users at some point in the future.