It's emacs(and xemacsand jed) -- not Ubuntu -- that creates those files. It's a copy of the previous version of the file your editing, ie. the file as it was before you made changes and used "save" to save them. I admit it's often a bother, but it's certainly useful if you've make changes you later regret.
It can be turned off manually -- and you can turn it off by default in the .emacsrc file.
Personally I run a cron-job that removes back-ups that hasn't been accessed for more than a week.
emacs also make files called something "#filename#" and ".#filename#". This file contained unsaved changes -- ie. things you've done since last save. The file is periodically updated as you edit a file. If emacs (or your computer) crashes, you'll be asked to continue with the "recovered buffer" if you try to edit the file again. (These files I also periodically "cron-away".)
My advice, keep letting emacs make back-ups... sooner or later you'll regret it if you don't.