I agree with C.S.Cameron, and I will add some details and links.
Steps to more space for persistence:
Create a home-rw persistence file up to 4GB and move your current home from casper-rw to this partition in your current persistent live system.
- Boot from another drive (or the same one, but live-only without the boot option
persistent).
- Create directories to become mountpoints.
- Mount
casper-rw and home-rw, for example to the mountpoints /media/casper-rw and /media/home-rw.
Then you can use rsync to copy /home like so (first a dry run, then remove the n and do the job),
sudo rsync -Havn /media/casper-rw/upper/home/ /media/home-rw # dry run
sudo rsync -Hav /media/casper-rw/upper/home/ /media/home-rw # do the job
The trailing slash in .../upper/home/ is important, see man rsync.
When you have checked that persistence is working from home-rw, you can remove the /media/casper-rw/upper/home directory tree.
Create a new persistent live system with a partition labeled casper-rw for persistence (and if you wish also a partition labeled home-rw.
See these links: mkusb and mkusb/persistent

Install Ubuntu (like into an internal drive) but to a USB drive,
See this link: Boot Ubuntu from external drive
If you intend to keep the Ubuntu system on the USB drive up to date, and/or if you need to install a lot of program packages, it is better to install Ubuntu (like into an internal drive) but to a USB drive. Such a system is portable between computers, but not as portable as a live or persistent live system.
It can be a good idea to install an Ubuntu community flavour with a light desktop environment, Lubuntu, Ubuntu Budgie, Ubuntu MATE or Xubuntu, because the communication is slower to a USB pendrive than to an internal hard disk drive or solid state drive. (A USB 3 SSD is fast alternative.)