-
Notifications
You must be signed in to change notification settings - Fork 5
Snapshots testing
The unionmount test suite is being used to test upstream overlayfs. The test suite was previously enhanced to verify the correctness of Overlayfs snapshots and later adapted to verify the correctness of Overlayfs watch change tracking snapshots.
Following are instructions how to setup and run the test suite.
Clone the enhanced unionmount-testsuite sources from github:
# git clone -b ovl-snapshot-watch https://github.com/amir73il/unionmount-testsuite.git # cd unionmount-testsuite
To setup an overlayfs watch over tmpfs, run:
# mkdir -p /base # ./run --sn --set-up # mount |tail -3 lower_layer on /base type tmpfs (rw,relatime) overlay on /base/s/0 type overlay (rw,relatime,lowerdir=/base/l,upperdir=/base/0/u,workdir=/base/0/w,index=on,watch=sb) lower_layer on /base/m type tmpfs (rw,relatime)
- /base is the local file system
- /base/l contains an initial setup of files and dirs for the different tests
- /base/m is a bind mount to /base/l where the tests perform file operations
- /base/0/w/index is where the changed directories are recorded
- /base/b/ will contains backups of the setup in /base/l before the tests begin
- /base/s/0 is the overlayfs watch mount, where changed subtree can be queried
To execute all unionmount tests with a single overlayfs watch, run:
# ./run --sn --verify *** *** ./run --sn --ts=0 open-plain *** TEST open-plain.py:10: Open O_RDONLY ./run --open-file /mnt/a/foo100 -r -R :xxx:yyy:zzz ./run --open-file /mnt/a/foo100 -r -R :xxx:yyy:zzz ... TEST rename-pop-dir.py:148: Rename dir and rename back ./run --rename /base/m/a/dir111 /base/m/a/no_dir111 ./run --mkdir /base/m/a/no_dir111/pop/new 0755 ./run --rename /base/m/a/no_dir111 /base/m/a/dir111 ./run --open-file /base/m/a/dir111 -r -d ./run --open-file /base/m/a/no_dir111 -r -d -E ENOENT ./run --open-file /base/m/a/dir111/a -r ./run --open-file /base/m/a/dir111/pop/new -r -d [ 1109.891407] sh (3152): drop_caches: 3 (cd /base/l; diff -qr --no-dereference a /base/b/0/a 2>&1 | grep -v Only.in./base/b) >/tmp/diff && grep '^Only in ' /tmp/diff | sed 's/^Only in \(\S*\):.*$/\1/p' >/tmp/only && grep '^Files ' /tmp/diff | sed 's/^Files \(\S*\)\/\S* and.*$/\1/p' >/tmp/differ && cat /tmp/differ /tmp/only | sort -u >/tmp/changed && (cd /base/s/0; du -x a 2>/tmp/duerr) | cut -f2 > /tmp/du && sed "s/^du: cannot access .\(\S*\).: Stale file handle$/\^\1/p" /tmp/duerr >/tmp/stale && grep -v -x -f /tmp/du /tmp/changed | grep -v -f /tmp/stale; test $? -ne 0
du -x on the overlayfs watch mount is used as a method to list the changed parts on the subtree. The clumsy shell script ran after each test verifies that all directories that differ between the subtree a and its backup have been recorded in the change tracking snapshot.
To execute a specific test (e.g. mkdir) with a up to 10 concurrent overlayfs watches, run:
# ./run --sn=10 --verify --recycle mkdir *** *** ./run --sn=10 --ts=0 mkdir *** TEST mkdir.py:10: Create directory ... TEST mkdir.py:114: Create directory over dangling sym ./run --mkdir /base/m/a/pointless110 0755 -E EEXIST ./run --mkdir /base/m/a/pointless110 0755 -E EEXIST [ 1945.418978] sh (3818): drop_caches: 3 (cd /base/l; diff -qr --no-dereference a /base/b/0/a 2>&1 | grep -v Only.in./base/b) >/tmp/diff && grep '^Only in ' /tmp/diff | sed 's/^Only in \(\S*\):.*$/\1/p' >/tmp/only && grep '^Files ' /tmp/diff | sed 's/^Files \(\S*\)\/\S* and.*$/\1/p' >/tmp/differ && cat /tmp/differ /tmp/only | sort -u >/tmp/changed && (cd /base/s/0; du -x a 2>/tmp/duerr) | cut -f2 > /tmp/du && sed "s/^du: cannot access .\(\S*\).: Stale file handle$/\^\1/p" /tmp/duerr >/tmp/stale && grep -v -x -f /tmp/du /tmp/changed | grep -v -f /tmp/stale; test $? -ne 0 (cd /base/l; diff -qr --no-dereference a /base/b/1/a 2>&1 | grep -v Only.in./base/b) >/tmp/diff && grep '^Only in ' /tmp/diff | sed 's/^Only in \(\S*\):.*$/\1/p' >/tmp/only && grep '^Files ' /tmp/diff | sed 's/^Files \(\S*\)\/\S* and.*$/\1/p' >/tmp/differ && cat /tmp/differ /tmp/only | sort -u >/tmp/changed && (cd /base/s/1; du -x a 2>/tmp/duerr) | cut -f2 > /tmp/du && sed "s/^du: cannot access .\(\S*\).: Stale file handle$/\^\1/p" /tmp/duerr >/tmp/stale && grep -v -x -f /tmp/du /tmp/changed | grep -v -f /tmp/stale; test $? -ne 0 (cd /base/l; diff -qr --no-dereference a /base/b/2/a 2>&1 | grep -v Only.in./base/b) >/tmp/diff && grep '^Only in ' /tmp/diff | sed 's/^Only in \(\S*\):.*$/\1/p' >/tmp/only && grep '^Files ' /tmp/diff | sed 's/^Files \(\S*\)\/\S* and.*$/\1/p' >/tmp/differ && cat /tmp/differ /tmp/only | sort -u >/tmp/changed && (cd /base/s/2; du -x a 2>/tmp/duerr) | cut -f2 > /tmp/du && sed "s/^du: cannot access .\(\S*\).: Stale file handle$/\^\1/p" /tmp/duerr >/tmp/stale && grep -v -x -f /tmp/du /tmp/changed | grep -v -f /tmp/stale; test $? -ne 0 (cd /base/l; diff -qr --no-dereference a /base/b/3/a 2>&1 | grep -v Only.in./base/b) >/tmp/diff && grep '^Only in ' /tmp/diff | sed 's/^Only in \(\S*\):.*$/\1/p' >/tmp/only && grep '^Files ' /tmp/diff | sed 's/^Files \(\S*\)\/\S* and.*$/\1/p' >/tmp/differ && cat /tmp/differ /tmp/only | sort -u >/tmp/changed && (cd /base/s/3; du -x a 2>/tmp/duerr) | cut -f2 > /tmp/du && sed "s/^du: cannot access .\(\S*\).: Stale file handle$/\^\1/p" /tmp/duerr >/tmp/stale && grep -v -x -f /tmp/du /tmp/changed | grep -v -f /tmp/stale; test $? -ne 0
Every time that a new directory is created the testsuite mounts a new overlayfs watch at /base/s/N creates a backup at /base/b/N.
After the test, the clumsy script verifies the correctness of all the change tracking snapshots that were created during the test.
Following example creates a test volume named 'lower_layer' formatted with XFS:
# lvcreate -n lower_layer -L 5g ssd
# mkfs.xfs /dev/mapper/ssd-lower_layer
meta-data=/dev/mapper/ssd-lower_layer isize=512 agcount=4, agsize=327680 blks
= sectsz=512 attr=2, projid32bit=1
= crc=1 finobt=1, sparse=0, rmapbt=0, reflink=1
...
# echo /dev/mapper/ssd-lower_layer /base xfs noauto >> /etc/fstab
NOTE that the name of the volume must end with 'lower_layer' as this is how the testsuite setup recognizes this mount.
To cleanup old tmpfs mounts and setup an overlayfs watch over the newly created test volume, run:
# ./run -c # ./run --sn --set-up # mount |tail -3 /dev-lower_layer on /base type xfs (rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota) overlay on /base/s/0 type overlay (rw,relatime,lowerdir=/base/l,upperdir=/base/0/u,workdir=/base/0/w,index=on,watch=sb) /dev-lower_layer on /base/m type xfs (rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota)
The testsuite setup recognizes the fstab entry for /base and mounts it as the local filesystem instead of mounting a tmpfs.