I like 'Disks' (gnome-disk-utility) and its ability to create backups of system partitions.
But at some point I read on the internet about a program that would create such images that are not as large as the whole partition, but only as large as the files on it. Now I cannot find it again.
Is there such an utility?
The following answer uses a TUI, not a GUI solution:
Depending on your use-case (I use it to make system back-ups) you can use CloneZilla Live. It can create:
The main advantages are:
On top of all these goodies, it can copy MBRs as well.
If a cold system back-up is not what you're looking for, you can still use the technology that CloneZilla is based upon and that is partclone.
Achieving smaller disk images is usually done by ways of not copying zero outed blocks from the initial file system in the image. The concept is called sparse space in files or puching holes in files, and your host file system has to support it. a bit command line but try this:
sudo apt-get install virt-manager qemu-utils
And then once it's install you can use :
qemu-img convert -f raw -O raw /dev/sdX /home/<user>/Ubuntu.img
This will create a sparse ( zeros in the file aren't written) image of your disk.
where -f = format, -O output format. sdX the disk ( can use partitions as well).
The program has advanced features such as snapshots and different output formats (vdi, vmdk, etc).
Your place where you save the image has to support sparse files, or else the size will be just as big, since zeros in the partition will be written to disk. And to make the most of it, do run a program such as sfill or zerofree found in apt on the partition, because free space does not always mean zero space, and you want to maximize the amount of zero space on the file system.
Bonus ninja points. Since linux thinks of disks as "files", then you could also use plain cp, with the condition that your output partition where you park the file knows sparse files.
sudo cp --sparse=always /dev/sdX /home/<user>/Ubuntu.img
Which will create a raw copy by means of cp..
Examples of filesystems that support sparse : ext4, btrfs, xfs, ntfs ( ntfs-3g however might not allow sparse writes).
Examples of filesystems that do not : fat32.
As there doesn't seem to be any program that all by itself would satisfy the two conditions:
1) Have a 'Disks'-like GUI;
2) Create disk-image as large as the space used,
I was thinking about a rather simple workaround, but one involving two GUI programs, very common in Ubuntu:
One would be the same 'Disks' (gnome-disk-utility), used to to create a backup image of a given partition, and the second would be Gparted, to shrink the partition as much as possible before creating its backup image, and to resize it back to original after that.
I see no inconvenient in a such operation: shrinking a partition down to minimum would only take a few minutes (about a minute for 10GB) and would create an empty space to the right. After making a backup image of the disk with 'Disks', using Gparted the partition can be resized back to the right into the empty space up to its initial state, and that would take only a few seconds.
Tested on a partition with an Ubuntu installation:
shrink partition with Gparted as much as possible
backup image of that partition with Disks
restore image with Disks onto a partition larger than the image; that partition should be un-mounted first: How to correctly restore system partition (iso image) with 'Disks'.
Edit: This will not work well if files deleted from the filesystem are still present on the partition, so first it's necessary to wipe the free space on the target partition:
dd if=/dev/zero of=<path_to_partition_volume>/file.dd && rm <path_to_partition_volume>/file.ddThen:
You can read the whole partition block by block with dd and output everything to a compression utility like gzip. Most likely any compression tool, even using the fastest/lowest compression settings, will compress all the free space during the process.
With gzip:
sudo dd if=/dev/sdX | GZIP=-1 gzip >/path/to/backup.gz
(GZIP=-1 temporarily sets the environment variable GZIP to -1 to achieve the fastest/lowest compression)
To restore a backup:
gzip -d /path/to/backup.gz >/dev/sdX
How about using ddrescue with -S option?
ddrescue -S /dev/sdXY /<path_to_backup>/backup.img
This will clone each block but zero blocks found in block device /dev/sdXY to /<path_to_backup>/backup.img
It can't be used to make a copy to a smaller drive. When I have a 64 Gb drive that is only 10% filled with data. I would like to be able to copy that to a 6 Gb imagefile, that I can copy on its turn to a 16Gb drive and than expand it to 16 gb
The only possibility I know now is use GParted to shrink the original, than copy it and then resize both. A very risky operation, since you need to shrink and resize the original.
You could use rsync to backup just the files on the partition to another partition. Or tar to create a compressed archive containing all files.
Both probably have (multiple) GUIs, but I've never used those myself.
Clonezilla can actually be quite friendly. The live CD doesn't require any command line usage - I'd be quite happy to call it a GUI even though it doesn't use the mouse - and I think the defaults are OK for a lot of cases. The problem is something you didn't explicitly call out: it doesn't seem to be designed as something you can install on an arbitrary linux system. It's packaged for Debian / Ubuntu specifically, and the Live CD is the easiest way to run it.
Clonezilla / partclone use filesystem-aware cloning and use their own compressed image format.
There's another non-GUI possibility to generate simple (mountable) sparse files, that doesn't rely on detecting zero sectors.
virt-sparsify /dev/your-block-device backup.img
If /tmp is a ramdisk it will warn that it potentially needs more space for temporary files. So you might need to add the option --tmp /var/tmp.
