Solution:
In terminal, run gedit ~/.local/share/applications/mimeapps.list. In the text document that opens, add the line application/octet-stream=gvbam.desktop; underneath "[Added Associates]", which should be at the top. This should force .gbc files, which Ubuntu recognizes as octet-streams, to open using VBA-M.
You will also have to tell VBA-M to load a rom upon launching by using the following steps:
- In terminal, run
gksudo gedit /usr/share/applications/gvbam.desktop
- In the file that opens, replace
Exec=gvbam with Exec=gvbam %U
Following this, you will need to run sudo update-desktop-database in terminal to "apply" the edits.
When you click "Open with other application..." in the right-click menu, and click "Show other applications", VBA-M should now be in the list, and the game should open correctly.
If file is not recognized as a GameBoy Color Rom or isn't opening, rename the extension. For example, if the extension is ".gbc" (i.e. GameBoy Color), renaming it to ".gb" (i.e. GameBoy) should make it recognized as a GameBoy Rom.
How I came to my conclusion
In terminal, run gksudo gedit /usr/share/applications/gvbam.desktop. In the "Exec=gvbam" line, add %U afterwards. So in the end, it should the whole line should be Exec=gvbam %U.
While this did add VBA-M to the "Other Applications" list, (and thus allow me to set VBA-M as the default program for .gbc's) the game doesn't actually run; in fact, the VBA-M window completely fails to show up anywhere. I think that this is because of how VBA-M currently works; in a terminal, I ran gvbam "Pokemon Gold.gbc", and it threw the following error:
The program 'gvbam' received an X Window System error.
This probably reflects a bug in the program.
The error was 'BadAlloc (insufficient resources for operation)'.
(Details: serial 474 error_code 11 request_code 154 minor_code 3)
(Note to programmers: normally, X errors are reported asynchronously;
that is, you will receive the error a while after causing it.
To debug your program, run it with the --sync command line
option to change this behavior. You can then get a meaningful
backtrace from your debugger if you break on the gdk_x_error() function.)
To further test this, I downloaded another rom, and I chose The Legend of Zelda: A Link to the Past. Having already done the above instructions to add VBA-M to the application list, I chose it as default for the Zelda game, which was a .gba file; the game loaded fine. I noticed in the properties of the files, the .gbc's type was unknown (application/octet-stream), while the .gba's type was Game Boy Advance ROM (application/x-gba-rom); obviously, something else was missing in the association of GameBoy Color roms, which wasn't missing with GameBoy Advance roms.
I tried adding gbc as an associated filetype with application/octet-stream, and that seemed to fix the issue of the .gbc file not opening. However, I then removed it, and tried opening the .gbc again, and it still worked; I'm not sure what I did, but now it works.
Original Answer
This got an upvote, so since someone thought this was good, I'm leaving it here for anyone else who cares. That being said, the information above is probably more relevant to the question.
To start out, you must be sure that the MIME type for your file extension does not already exist. Open a terminal and enter:
grep 'gbc' /etc/mime.types
If there was no output, or the output given did not include your extension, you must add a MIME type.
In a terminal, open the mime.types file by running gksudo gedit /etc/mime.types. In this file, add the following line to the "application" section:
application/gbc gbc
There should be additional spaces between the two gbc's - just make sure to line the second gbc up with the other extensions. I would show, but I'm not sure how to do this on askubuntu.
Sources:
