The correct place to put xmodmap ~/.Xmodmap (so that your changes are read at startup) is in ~/.xinitrc (see explanation below), although you could alternately place the xmodmap command as a startup item (/usr/bin/xmodmap /home/user/.Xmodmap). I can say that for 12.04 (and presumably 12.10) that these two alternatives definitely do work, as either way they are read after you log in with lightdm. (I changed Return to Right Shift, and the change only took effect after entering my password and logging in).
Please note: if you are on Xfce, it is noted on the official Xfce FAQ that you may have to create a startup item instead of using ~/.xinitrc, and that you might have to delay the execution so the xmodmap changes are not overwritten by setxkbmap. You can use a delay to achieve this in your startup entry:
/bin/bash -c "sleep 20; /usr/bin/xmodmap /home/$USER/.Xmodmap"
(For the startup entry it is necessary to specify absolute paths)
Each time setxkbmap is run to change keyboard layout, you will have to run xmodmap ~/.Xmodmap again in the terminal.
Creating ~/.xinitrc
Create a file in your home folder called .xinitrc with
touch .xinitrc
and place in it:
xmodmap ~/.Xmodmap
Your Xmodmap file should not have entries in the xmodmap -e " " style as that is used in the terminal for temporarily setting the values. It must have entries in this format:
keycode 65 = Tab
keycode 23 = Space
It is often confusing which of the various X files (such as .xprofile, .xinitrc, etc) are read at startup, but .xinitrc does seem to be still read under 12.04, whereas .Xmodmap or .xinputrc are not read by themselves.