The easiest solution would be to install Ubuntu on the external SSD.
The "problem" is that Linux does not install programs into a single directory, so, when you install an application it installs pieces in various locations on the hard drive.
See http://brajeshwar.com/2008/filesystem-file-organization-in-linux/
So, generally what you "normally" would do is, as a part of the installation, partition the ssd and mount the various partitions at various locations such as /home or /usr or /var
So you could reinstall or move part of the file system, such as /usr to your ssd. Moving will take longer and be more involved then simply re-installing, making a partition or partitions such as /var and /usr on the ssd. The installer will do this for you.
To move see How can I store /var on a separate partition?
Other options : How to use second HDD as program installation drive path
Your last option would be to compile from source. When compiling you specify the location, again you would make a partition on the ssd and mount it at /usr/local. When compiling you use the --prefix option
./configure --prefix=/usr/local
See https://stackoverflow.com/questions/8902698/linux-configure-make-prefix
So, if you did not understand what any of that means, you are best off installing Ubuntu onto the ssd.