PyPI Cache Now Live

August 17, 2023, Joachim Folz

After an extended testing period, our local PyPI cache is now live.

What’s different #

We applied the environment variables mentioned in our previous post to all newly opened shells on head nodes and job container environments. This means pip install commands now hit our cache instead of the PyPI servers.

Possible side effects #

In addition to the package files themselves, requests to the package index are also cached for 30 minutes to make the cache more responsive. That means new versions may appear up to 30 minutes after their release on PyPI.

Container images can define environment variables. If PIP_INDEX_URL, PIP_TRUSTED_HOST, or PIP_NO_CACHE is set by the container, the cache will not be used. However, we cannot distinguish between unset variables and variables set to empty string (instructing pip to use default values), and the cache will be used in that case.

Please report problems #

Since the cache has been running for over a year at this point, we don’t expect any major issues. Indeed, we have not observed any errors after 24 hours in production. However, please continue to report any problems you experience.