Merge branch 'dev' of github.com:facebook/zstd into dev

Author: Cyan4973

contrib/linux-kernel/README.md

@@ -1,6 +1,6 @@
 # Linux Kernel Patch
 
-There are two pieces, the `zstd_compress` and `zstd_decompress` kernel modules, and the BtrFS patch.
+There are three pieces, the `zstd_compress` and `zstd_decompress` kernel modules, the BtrFS patch, and the SquashFS patch.
 The patches are based off of the linux kernel master branch (version 4.10).
 
 ## Zstd Kernel modules
@@ -22,14 +22,15 @@ The patches are based off of the linux kernel master branch (version 4.10).
 
 * The patch is located in `btrfs.diff`.
 * Additionally `fs/btrfs/zstd.c` is provided as a source for convenience.
-* The patch seems to be working, it doesn't crash the kernel, and compresses at speeds and ratios athat are expected.
-  It can still use some more testing for fringe features, like printing options.
+* The patch seems to be working, it doesn't crash the kernel, and compresses at speeds and ratios that are expected.
+  It could still use some more testing for fringe features, like printing options.
 
 ### Benchmarks
 
 Benchmarks run on a Ubuntu 14.04 with 2 cores and 4 GiB of RAM.
 The VM is running on a Macbook Pro with a 3.1 GHz Intel Core i7 processor,
 16 GB of ram, and a SSD.
+The kernel running was built from the master branch with the patch (version 4.10).
 
 The compression benchmark is copying 10 copies of the
 unzipped [silesia corpus](http://mattmahoney.net/dc/silesia.html) into a BtrFS
@@ -50,3 +51,39 @@ See `btrfs-benchmark.sh` for details.
 | zstd 9    | 2.92              | 43 MB/s           | 406 MB/s            |
 | zstd 12   | 2.93              | 21 MB/s           | 408 MB/s            |
 | zstd 15   | 3.01              | 11 MB/s           | 354 MB/s            |
+
+
+## SquashFS
+
+* The patch is located in `squashfs.diff`
+* Additionally `fs/squashfs/zstd_wrapper.c` is provided as a source for convenience.
+* The patch has been tested on a 4.10 kernel.
+
+### Benchmarks
+
+Benchmarks run on a Ubuntu 14.04 with 2 cores and 4 GiB of RAM.
+The VM is running on a Macbook Pro with a 3.1 GHz Intel Core i7 processor,
+16 GB of ram, and a SSD.
+The kernel running was built from the master branch with the patch (version 4.10).
+
+The compression benchmark is the file tree from the SquashFS archive found in the
+Ubuntu 16.10 desktop image (ubuntu-16.10-desktop-amd64.iso).
+The compression benchmark uses mksquashfs with the default block size (128 KB)
+and various compression algorithms/compression levels.
+`xz` and `zstd` are also benchmarked with 256 KB blocks.
+The decompression benchmark is timing how long it takes to `tar` the file tree
+into `/dev/null`.
+See `squashfs-benchmark.sh` for details.
+
+| Algorithm      | Compression ratio | Compression speed | Decompression speed |
+|----------------|-------------------|-------------------|---------------------|
+| gzip           | 2.92              |   15 MB/s         | 128 MB/s            |
+| lzo            | 2.64              |  9.5 MB/s         | 217 MB/s            |
+| lz4            | 2.12              |   94 MB/s         | 218 MB/s            |
+| xz             | 3.43              |  5.5 MB/s         |  35 MB/s            |
+| xz 256 KB      | 3.53              |  5.4 MB/s         |  40 MB/s            |
+| zstd 1         | 2.71              |   96 MB/s         | 210 MB/s            |
+| zstd 5         | 2.93              |   69 MB/s         | 198 MB/s            |
+| zstd 10        | 3.01              |   41 MB/s         | 225 MB/s            |
+| zstd 15        | 3.13              | 11.4 MB/s         | 224 MB/s            |
+| zstd 16 256 KB | 3.24              |  8.1 MB/s         | 210 MB/s            |

contrib/linux-kernel/squashfs-benchmark.sh

@@ -0,0 +1,39 @@
+# !/bin/sh
+set -e
+
+# Benchmarks run on a Ubuntu 14.04 VM with 2 cores and 4 GiB of RAM.
+# The VM is running on a Macbook Pro with a 3.1 GHz Intel Core i7 processor and
+# 16 GB of RAM and an SSD.
+
+# $BENCHMARK_DIR is generated with the following commands, from the Ubuntu image
+# ubuntu-16.10-desktop-amd64.iso.
+# > mkdir mnt
+# > sudo mount -o loop ubuntu-16.10-desktop-amd64.iso mnt
+# > cp mnt/casper/filesystem.squashfs .
+# > sudo unsquashfs filesystem.squashfs
+
+# $HOME is on a ext4 filesystem
+BENCHMARK_DIR="$HOME/squashfs-root/"
+BENCHMARK_FS="$HOME/filesystem.squashfs"
+
+# Normalize the environment
+sudo rm -f $BENCHMARK_FS 2> /dev/null > /dev/null || true
+sudo umount /mnt/squashfs 2> /dev/null > /dev/null || true
+
+# Run the benchmark
+echo "Compression"
+echo "sudo mksquashfs $BENCHMARK_DIR $BENCHMARK_FS $@"
+time sudo mksquashfs $BENCHMARK_DIR $BENCHMARK_FS $@ 2> /dev/null > /dev/null
+
+echo "Approximate compression ratio"
+printf "%d / %d\n"                                                             \
+  $(sudo du -sx --block-size=1 $BENCHMARK_DIR | cut -f1)                       \
+  $(sudo du -sx --block-size=1 $BENCHMARK_FS  | cut -f1);
+
+# Mount the filesystem
+sudo mount -t squashfs $BENCHMARK_FS /mnt/squashfs
+
+echo "Decompression"
+time sudo tar -c /mnt/squashfs 2> /dev/null | wc -c > /dev/null
+
+sudo umount /mnt/squashfs