Let’s talk about compression. As a Linux user, you’re probably already familiar with longtime staples like gzip, bzip2, and xz. They’ve been around for a while, and most everyone has at least stumbled across them by now, if they’re not actively using them on a regular basis.
As if there weren’t enough already, there’s a newcomer in the compression arena. Zstandard (often abbreviated to zstd) was released in 2015 and since then has risen sharply in popularity. Zstandard was created by a developer at Facebook. Since then, Zstandard has been continuously used by the company to compress large amounts of data.
In the Linux world, official adoption of new standards can be rather slow, but Zstandard is seeing big support by distros such as Arch Linux, which switched from xz to Zstandard for package compression in the official repository back in 2019.
I hear ya. “Why use Zstandard? I already use gzip (or xz or 7-zip) compression.” Well, zstd is fast. Real fast. In our compression benchmark tests, Zstandard was significantly faster than gzip, bzip2, xz, 7-zip, and a couple other compression methods. Check the results below.
That’s where zstd excels – speediness. It only took 43 seconds to compress 1350 MB into 934 MB. Contrast this with gzip, which took a full extra minute to do the same job, and still couldn’t reduce the size as drastically.
Now, Zstandard is not the new answer to everything. Even when using its maximum compression level setting, it can’t match the size reduction offered by xz and 7-zip. But hey, Zstandard cares more about fast results, and I’m sure many of you do as well.
Install Zstandard on Linux
Your Linux distribution may already have Zstandard installed by default. If not, use the appropriate command below to install it with your system’s package manager.
Ubuntu, Debian, and Linux Mint:
$ sudo apt install zstd
Fedora, AlmaLinux, CentOS, and RHEL:
$ sudo dnf install zstd
Arch Linux and Manjaro:
$ sudo pacman -S zstd
How to Use Zstandard on Linux
Check out some of the commands below to see how to use Zstandard compression on Linux, which will involve the
zstd command and tar command.
Example 1. Zstandard uses the
.zst file extension. You can create a compressed zstd archive of a single file by using the following syntax.
$ zstd file.txt file.txt :144.83% ( 29 => 42 bytes, file.txt.zst) $ ls file.txt file.txt.zst
Example 2. To decompress a zstd archive, use the
$ zstd -d file.txt.zst
You could also use the
unzstd command instead.
$ unzstd file.txt.zst
Example 3. If you want to compress a directory, or combine multiple files into a single archive, you’ll need to use
tar to create an archive and then compress it with zstd. You’ll need to add the
--zstd option, along with any other flags you choose.
$ tar --zstd -cf archive.tar.zst /home/linuxnightly
Alternatively, we could use the
a option with the
tar command, which will choose the correct compression method based on the file extension (
zst in this case) specified.
$ tar acf archive.tar.zst /home/linuxnightly
Example 4. Use the following
tar command to open a Zstandard tarball.
$ tar --zstd -xf archive.tar.zst
a option can save us a few keystrokes again.
$ tar axf archive.tar.zst
Zstandard Compression Options
We’ve already covered the main
tar commands that you need to know for Zstandard compression. However, there are more options that can be used to control the compression speed and efficiency of Zstandard. See the examples below to learn about them.
Example 1. Zstandard’s default compression level is 3, but any setting between 1-19 can be set. A higher setting will yield a smaller compressed archive, at the cost of a slower compression speed. The following command would set zstd to use a compression level of 8.
$ zstd -8 file.txt
To use this compression setting with
tar, you’ll need to use the
-I option and specify the
zstd command and its extra options you wish to use.
$ tar -I 'zstd -8' -cf archive.tar.zst /home/linuxnightly
Example 2. Zstandard also features ultra compression levels 20-22, which can be accessed with the
--ultra option. Here’s how to get the maximum compression level from zstd.
$ zstd --ultra -22 file.txt
You can achieve Zstandard ultra compression on directories or multiple files by using the following
$ tar -I 'zstd --ultra -22' -cf archive.tar.zst /home/linuxnightly
Example 3. Zstandard’s
--fast option uses ultra-fast compression levels. You need to specify a number with the option. The higher the number, the quicker the compression will be, at the expense of some compression ratio. By default, just using
--fast (without a number) implies level 1.
$ zstd --fast=10 file.txt
And here’s the corresponding
tar command to utilize Zstandard’s ultra-fast compression setting.
$ tar -I 'zstd --fast=15' -cf archive.tar.zst /home/linuxnightly
Zstandard Environment Variables
Zstandard has two environment variables that users can set in order to control default zstd compression levels and the number of threads used during compression. Setting these to your preferred values will save you some keystrokes when executing the
ZSTD_CLEVEL controls default compression level for zstd.
ZSTD_NBTHREADS controls the number of threads used during zstd compression.
You can set these values system-wide by editing the
/etc/environment file and appending:
These variables would tell zstd to use compression level 8 by default, and use up to 6 threads simultaneously. Adjust to your own needs.
Hint: See how many threads your system has available by executing:
$ nproc --all
Zstandard is the newest and fastest compression tool on Linux. In this guide, you learned how to install it and use
tar commands to utilize Zstandard. We’ve shown you all the most essential commands that you’ll need. If you think we’ve missed one, feel free to let us know in the comments below.
3 thoughts on “How to Use Zstandard Compression on Linux (With Commands)”
This popped up in a project I’m working on. Thanks for the complete, but brief, rundown on it.
Very nice, short tutorial.
One thing I would add is putting ZSTD_CLEVEL and ZSTD_NBTHREADS in environmental variables to be able to overwrite default settings of compression level and usable thread number system-wide. without the need of specifying it every time in command/script.
I was updating outdated backup scripts after previous IT Tech and I’ve switched from single core tar gzip to multicore tar zstd with custom compression level and thread count. My archives went from 37GB to 27GB (compression level=12) and time went down from 75 minutes to 8 minutes (around 60000 files, thread count 6), which is a big improvement.
Thanks for your suggestion on zstd environment variables. We’ve added a section for it to the tutorial.