Many times I get this dialog when opening some files. It asks me to choose a program. I don't know where all programs reside and what to select? Where to find installed applications in Ubuntu?

You need to choose the appropriate program to open that file. Most of the programs on Ubuntu are installed in /usr/bin/ directory. So, once you click "Choose" and the window pops up for you to navigate to the program, navigate through /usr/bin/ and find the appropriate program. For example, if you have clicked on a torrent/magnet link, you would select transmission-gtk. To open an application installed in a custom directory, you need to navigate through that directory and select the program.
If you can't find the application in /usr/bin/; then the other appropriate directories to look out for are /bin/, /usr/share/, /usr/local/, /opt/. See the following question for further details: