Software problem: Deleting Btrfs subvolumes

Internet is no longer full of Btrfs horror stories so I decided to use Btrfs when I reinstalled my desktop computer with NixOS. It has some nice features, like subvolumes, compression and cooperation with Podman. I still prefer ZFS but due to its weird license it’s not included in Linux and one can fall into trouble when using it as the root file system and something goes wrong with a kernel module upgrade (as happened to me couple of times in the past). Btrfs doesn’t have this problem so it’s preferable for me on desktop computers.

Now I experienced the first problem with Btrfs. When I tried to remove some containers and prune Podman images, I got complaints about files that couldn’t be deleted from the storage. This has led me to the fact that Btrfs permits non-root users to create subvolumes but not to delete them:

$ btrfs subvolume create $HOME/test
Create subvolume '/home/pdm/test'

$ btrfs subvolume delete $HOME/test
WARNING: cannot read default subvolume id: Operation not permitted
Delete subvolume (no-commit): '/home/pdm/test'
ERROR: Could not destroy subvolume/snapshot: Operation not permitted
WARNING: deletion failed with EPERM, send may be in progress

Huh? At least root can still delete the subvolume:

$ sudo btrfs subvolume delete $HOME/test
Delete subvolume (no-commit): '/home/pdm/test'

And removing the volume by simply removing its directory also works, even for a non-root user. But it looks like Podman wants, not surprisingly, delete the volume. I’m not the first one being hit by this problem. It should be fixed but it still doesn’t work for me.

There is a mount option user_subvol_rm_allowed, unfortunately disabled by default, mostly for legacy reasons. After I had added it to my / and /home subvolumes, deletion of subvolumes started to work.

But not so Podman. I was no longer able to create or rebuild containers. For example:

$ podman run -it --rm debian
Error: unlinkat /home/pdm/.local/share/containers/storage/btrfs/subvolumes/6a7bcf44cc576f9ba66b978fc7472380ba0a8e5db318d547f6484acd3e49995e/bin: permission denied

Should I delete the subvolume? Let’s try:

$ btrfs subvol delete /home/pdm/.local/share/containers/storage/btrfs/subvolumes/6a7bcf44cc576f9ba66b978fc7472380ba0a8e5db318d547f6484acd3e49995e
WARNING: cannot read default subvolume id: Operation not permitted
Delete subvolume (no-commit): '/home/pdm/.local/share/containers/storage/btrfs/subvolumes/6a7bcf44cc576f9ba66b978fc7472380ba0a8e5db318d547f6484acd3e49995e'
ERROR: Could not destroy subvolume/snapshot: Permission denied

OK, still not all right, so I removed the subvolume as root. But then:

$ podman run -it --rm debian
Error: stat /home/pdm/.local/share/containers/storage/btrfs/subvolumes/6a7bcf44cc576f9ba66b978fc7472380ba0a8e5db318d547f6484acd3e49995e: no such file or directory

Oops. It looked like the most efficient way to get rid of the mess was to remove all the containers, all the subvolumes and to rebuild the containers again (fortunately, I haven’t had anything valuable in them yet).

After some more trouble (even podman system reset hadn’t worked), I ended up with removing ~/.local/share/containers/ completely and Podman finally started to work.

OK, mostly problems of category 1. The lesson learned is to always use user_subvol_rm_allowed with Btrfs.