Not directly answering your question, I would say hard-embedding them on top of your video is way too destructive. YouTube agrees with me; they don't edit the videos you annotate, they just show a deebrie above the video.
And you can do exactly what YouTube does with your HTML video. The difference is you need to do it. In short: load a list of annotations (their text, time, duration and position), listen to the playing event of the video, and the pop up divs over the video while it's playing.
There are HTML5 "players" out there (they're more frameworks around the <video> tag) but none I know of do this for you. JW Player probably has the most advanced solution but it's still in development and they claim their JS API "will" change before final release. You might need to change things as they do but it's still a base to work from.
http://developer.longtailvideo.com/trac/wiki/HTML5Overview