In a terminal, run the following:
gpg --export-secret-key -a > secretkey.asc
And on the other system, import the secret key with:
gpg --import secretkey.asc
Alternatively, if you've got ssh access to the other system you should be able to combine these two actions into a single command:
gpg --export-secret-key -a | ssh othermachine gpg --import -
Once the keyfiles have served their purpose, securely delete them:
shred secretkey.asc && rm secretkey.asc
or
shred --remove secretkey.asc
Make sure to shred and remove the key instead of using normal deletion. Additionally, instead of moving the keyfile with mv, copy it, then shred and remove the original. These methods will prevent an attacker from recovering the key through low-level bit inspection.