This is in principle how it is done:
<source src="song.ogg" type="audio/ogg"></source>
<source src="song.mp3" type="audio/mpeg"></source>
Your browser does not support the audio element.
I have not been able to get this HTML5 audio working on the new Windows Phone Internet Explorer. The visual player interface shows up, I can press the play button, but nothing happens. It might be that this has to do with a lack of support for the explicit control of the methods play(), but I have not yet done any further experiments. If Windows Phone is supposed to be second in the mobile market place by 2015, as suggested in press articles recently, then the Windows Phone developers have to go back to work and solve these issues quickly. Playing MP3s is nowadays essential for the success of a mobile phone, and HTML5 audio is the way to do it properly.
On my website I did implement a few things: first I made the sound to fade-in / fade-out, when the play/pause button is clicked. I always hate it when the sound suddenly starts or suddenly stops, and many of the commercial music distributors such as Amazon have implemented this fading, probably on their server side. Using the HTML5 for this is a bit problematic: the fading does not work very smoothly, as the volume change is apparently only applied when the next chunk of audio data is played, making the volume change a bit choppy. And when pausing it, there is a short audio gap, before I override the pause button and make it play again with a fade-out. Not very elegant, but works for now ok.
One important thing that I also have implemented is to have a more direct control over protecting the MP3 files. This would probably also work without HTML5 (I did not try), but using HTML5 gave me the opportunity to change how the MP3 files are accessed. Instead of directly putting the music filename into the src field of the audio tag, I put a link to another PHP script. In there, I read the file (hereby hiding the real filename and the location folder from the user) and stream it out. This allows to stream only a certain part of a file, which could be the first 30 seconds or a random part within the file. I also have implemented a kind of authentication scheme, which ensures that the file is accessed only through proper authorisation. This scheme is based on a "secret" calculation of a code, which needs to match a code that is associated with the MP3 file. Works fine so far, and I can even control selectively the file access. For example, those who come to my site through Facebook and have "liked" my facebook page, will get more free access to music. Hopefully this will add more fans!