ZStandard the new compression algorithm

• Facebook’s new compression algorithm
• Has very good performance numbers (speed) compared to gzip and the ’traditional’ unix tools
• Linux binaries are available for all popular distributions search for zstd compression.
• brew install zstd to install on Mac.

Comparison of file sizes with various compressions.

• zstd was run with -20 (highest compression)
• gzip with (-9)
• gzip was 7 times slower.

    -rw-r--r--  1 vivekv  staff   301M May 23 20:38 ipdata.sqlite3
    -rw-r--r--  1 vivekv  staff   103M May 23 20:38 ipdata.sqlite3.gz
    -rw-r--r--  1 vivekv  staff    66M May 23 20:38 ipdata.sqlite3.zst