apt will use whatever repositories you have added to collect packages. If you limit this to Canonical ones, everything you get has been tested and in the majority of situations and setups, works well.
They will, for the most part, not be the bleeding edge that your software packages of choice have to offer.
They will be as stable and as secure as the people who curate those repositories can manage and will have as few compatibility and dependencies issues as humanly possible.
You can add numerous 3rd party repositories to apt. Doing so will update apt's view of available packages and versions but can start to break dependencies.
If an installed package A is dependent on installed package B, but package C needs an older or new version of package B, you will get an error when trying to install package C. Forcing it may break package A as well.
You can usually rectify most broken packages using apt or dpkg, which apt is effectively a wrapper for.
If you start installing packages from non apt based sources, be that their source code or pre-compiled binaries, you are at the mercy of their requirements. If they have dependencies you don't meet, you will need to manually install them. Possibly via apt, possibly not. Blindly following installation instructions can install things into your system which can overwrite apt sourced packages, or parts of them, effectively breaking anything that requires that package to be at a certain version. Alternatively they can be installed to your user only, so other user accounts will struggle to make use of them.
Ultimately you can do whatever you want and are comfortable with.
You can always start over if you break it, so keep the USB drive somewhere safe.