YouTube provides an ActionScript API.
Using this, you could load the videos into Flash using their API and then have your Flash app create the annotations on a layer above the video.
Or, alternatively, if you want to stay away from creating something in Flash, using YouTube's JavaScript API you could draw HTML DIVs over the YouTube player on your web page. Just remember when you embed the player to have WMODE="transparent"
in the params list.
So using the example from YouTube:
<script type="text/javascript">
var params = { allowScriptAccess: "always" };
var atts = { id: "myytplayer", wmode: "transparent" };
swfobject.embedSWF("http://www.youtube.com/v/VIDEO_ID&enablejsapi=1&playerapiid=ytplayer",
"ytapiplayer", "425", "356", "8", null, null, params, atts);
</script>
And then you should be able to draw your annotations over the YouTube movie using CSS/DHTML.