I know that this is usually not the way to use AngularJS but I was wondering if what I want to achieve is doable with AngularJS. If this is not the recommended way could you ple
You could get hold of the controller, but that would make the whole communication between SVG and your app in-side-out. Instead invert how the application is composed: Let angular drive the app and wrap the svg using a directive into a reusable component as Dan mentioned.
btw using SVG and angular is not such a rare thing. check out this pretty awesome example: http://sullerandras.github.com/SVG-Sequence-Diagram/
You should look at creating an angularjs directive that wraps these objects. When data that is bound to angular scope changes, you need to do a scope.$apply(). This is a simplistic answer, take a look at the http://docs.angularjs.org/guide/directive doc.