Using GNOME Mplayer everytime I play a video, I get the annoying dialog pop-out in the screenshot below.

I'm using the proprietary NVIDIA driver and I fixed this by creating a symlink to the VDPAU lib:
ln -s /usr/lib/vdpau/libvdpau_nvidia.so.1 /usr/lib/libvdpau_nvidia.so
This has the benefit of actually enabling VDPAU rather than falling back to XV or X11. Of course you need a video card that supports VDPAU acceleration.
You can suppress all dialogues like this in mplayer variants by editing ~/.mplayer/config in the editor of your choice and somewhere in the file including really-quiet="1". I use plain mplayer (no GUI at all) but that shouldn't change things, this worked for me in GMplayer too. Here's my config:
[default]
vo=vdpau,xv,
vc=ffh264vdpau,ffmpeg12vdpau,
ao=pulse
really-quiet="1"
fs=1
xineramascreen=1
msglevel=all=0
showsubtitles=0
You probably don't want the vo and vc lines (I'm a Nvidia user) and you might not want any of the other settings. If I were you, I would just use this:
[default]
ao=pulse
really-quiet="1"
For Intel VGA, and x64 I have solved it with
sudo apt-get install i965-va-driver
sudo add-apt-repository ppa:nilarimogard/webupd8
sudo apt-get update
sudo apt-get install libvdpau-va-gl1
cd /usr/lib/x86_64-linux-gnu/vdpau/
sudo ln -s libvdpau_va_gl.so.1 libvdpau_i965.so.1
For Nvidia the package are
sudo apt-get install libvdpau1 vdpau-va-driver
Here is the solution to your problem:
Just try specifying a video output driver on the command line when you launch mplayer like this:
mplayer -vo xv video.wmv
You can get a list of "Available video output drivers" with the command mplayer -vo help.
The problem is that the selected (or default) video output driver was vdpau, and no one working on the mplayer packages and software cares enough about having the software package work with your configuration of hardware and software. But do not worry, you are now blessed with this answer, and the video output driver can be specified on the command line (e.g. -vo xv).
The order of configuration options that mplayer will scrawl before it finally decides to (only) try the (missing) vdpau driver is not so obvious, but I think it roughly goes like this,
~/.mplayer/config for the (user specific) default video output driver, if any/etc/mplayer/mplayer.conf for the (system specific) default video output driver, if anyIf the mplayer command you were using was compiled with the vdpau option and there are no default video output drivers specified in ~/.mplayer/config or /etc/mplayer/mplayer.conf (which is either a common occurrence or I am the only person on the planet earth who has ever had problems with a configuration option on a linux box) then it will fall back to the compiled default which in this case was vdpau.
Another similar issue is that mplayer may work fine while gmplayer gives this same error using the same options you just tried with mplayer. Well guess what? gmplayer uses yet another config file, ~/.mplayer/gui.conf, and it will default to the last video driver specified (or defaulted to in the compile options). So again, just specify a video output driver on the command line, or put one in one of the myriad of config files mentioned here.
However, a little setting can sort out the problem as follows;
x11 for the Video OutputYour problem should have been resolved.
Note: Everything before update was included because it may be useful for someone else, and was posted written either before testing or when I was lacking info on the situation.
If you are using an nVidia card, install the proprietary drivers.
Fglrx is the official (closed source) ATI/AMD driver, though there are also official open source drivers included in Ubuntu by default, and these work just as well with many (older, usually) cards. vdpau (sorry if I misspelled earlier) is only currently available on nVidia and (to my knowledge) Intel cards. MPlayer supports this API for video acceleration, but it is not needed to play videos. As I am not an avid MPlayer user (I use
TotemorVLC Media Playerto play videos and DVDs), I will have to do some more digging for the correct entry in the configuration file.I would not suggest compiling the libvdpau.so file, as it would quite honestly be useless in your configuration, and if you were to get an nVidia card with the proprietary drivers you would not need to install anything extra anyway.
One suggestion I would give you is to install MPlayer-gui
(temporarily) and see if when launching this allows you to disable the use of vdpau from the gui options. Then you could revert to the normal MPlayer and it should work without any further problems. If that doesn't work, then you might have to end up compiling :( to disable the use of vdpau (and to avoid having to buy a new graphics card lol).
UPDATE
Don't laugh, (okay, laugh at me, go ahead ;) - but I just opened Gnome-MPlayer for myself and in preferences, you can choose your video output module! All you need to do now is to choose one that works for you (vaapi should work on your graphics card).
Under the MPlayer tab (not shown), you can also add command line parameters to pass to MPlayer just in case.
Customary screen-shot :)
The Wikipedia page for vaapi should help you to understand most of the video acceleration methods in detail (if you follow the links recommended in the article).
This command line helped me: mplayer -demuxer mov filename.mov
Installing libvdpau-va-gl solved it for me, but it was a software solution.
Try setting export VDPAU_DRIVER=nvidia