Image Compression: Epilog
October 29, 2023, Joachim Folz
Our image compression saga concludes (for now) with a comparison
of image sizes before and after we applied our new settings
to all existing images in /enroot.
Final results #
Over the past week or so, we explored the effects of compression settings on the Squashfs images from which containers are created on our cluster. We found that we could save a substantial amount of storage space and noticeably speed up container creation times. See parts 1, 2, and 3 for more details.
We have now upgraded all existing Squashfs images in the
/enroot directory to our new compression settings
(block size 1M, zstandard level 15).
In total, there were 336 images, weighing in at about 2.08 TiB,
which turned into just 1.05 TiB, or roughly 50% savings.
There are, however, significant differences depending on how old
the image is.
The following table shows the new size per year.
| Year | New size |
|---|---|
| 2017 | 41.1% |
| 2018 | 38.8% |
| 2019 | 41.4% |
| 2020 | 32.8% |
| 2021 | 72.8% |
| 2022 | 72.8% |
| 2023 | 73.6% |
After our first post in 2021, we updated the image import process
to use lz4hc compression, though we still managed to save
about 27% for those images.
Meanwhile, compression was mostly absent for legacy images from
2020 and earlier, which we converted from our previous Docker-based
deployment.
Below is a histogram of new image sizes as percentage of the old size.
The two groupings of legacy and newer images are evident here as well.
Uprading your own images #
Our upgrade process only touched images in /enroot.
You can upgrade your custom images stored elsewhere with the
following command:
srun \
--container-image=slow.sqsh \
--container-save=fast.sqsh \
true
This loads slow.sqsh and creates the upgraded image fast.sqsh.
It uses the new default settings for users, which are block size 1M
and zstandard level 2.
If you wish to change these, you can do so by overwriting the
ENROOT_SQUASH_OPTIONS environment variable:
srun \
--export="ENROOT_SQUASH_OPTIONS=-b 1048576 -comp zstd -Xcompression-level 15" \
--container-image=slow.sqsh \
--container-save=fast.sqsh \
true
The example sets level 15, which is what the image import process on the cluster uses. Compression is very slightly better, but exports take more than ten times longer. You can get most of the effect at level 3, and anything above 6 or 7 is probably overkill for users. See our previous post for more details.