If you want to use Javascript, Dojo is a great way to go. It has a compact, cross-platform (SVG, VML, Canvas, Silverlight) vector graphics API that is very powerful. You can find it in dojo.gfx and dojox.gfx.
We've used this to build an interactive physics tutor that allows students to draw vectors, ellipses, etc (even append images) and perform all sorts of transformations on them. You can see what we've done at http://gideon.eas.asu.edu/web-UI/login.html --just login with any username.
I've taken a look at fabric.js and dojox.drawing does a lot of the same things. If you look at the tests in the toolkit (once you've got it its dojox/drawing/tests/) you find examples of everything from vector graphics to images to programmatically created shadows.