I'm using ubuntu 21.10, installed anew with zfs filesystem. If I delete some files, and then empty the trashcan, or move them to another (external) drive) this doesn't free up space on the drive. actually df shows: rpool/ROOT/ubuntu_zooqq5 35G 4,5G 31G 13% / where 35GB is the drive size.
Every time I add some file to the disk, the drive size decrease.
I recover space (and drive size goes up) by deleting zfs snapshots with a script, but how can this be right?
Maybe it's a bug?
Removed files are still referenced by past snapshots, so what you describe is expected behaviour.
As you already observed, the space remains used due to snapshots; only after deleting the last snapshot that references the deleted files will the corresponding space be available again for new files.
Note that df does not provide accurate values for free space in ZFS file systems. ZFS uses one or more disks (or partitions) to store pools that can contain one or more datasets (interpreted by df as filesystems), all of which share the pool's free space. Snapshots also fill up the pool (as part of datasets), further affecting the 'Size' df reports for an individual file system. ZFS is both a file system, a volume manager, and optionally manages RAID arrays; it would be analogous to, e.g., ext4 on LVM optionally on RAID.
To see how pools correspond to disks or partitions, you can use zpool status. In Ubuntu, you can find the listed disks or partitions under /dev/disk/by-uuid/ and /dev/disk/by-partuuid/ respectively. To see the free space in pools, you can list all pools via zpool list. You can list all ZFS datasets ("file systems") via zfs list, where the USED column indicates the space used by the dataset, which includes the files, child datasets and snapshots belonging to a specific dataset; AVAIL indicates the remaining space in the pool, and REFER indicates the size of the files specific to this file system, i.e., not snapshots or child filesystems.
