I am working on an enterprise application development in ASP.NET MVC3. Of-course I have different master layouts and multiple views.
My concerns
<You need to think about reducing your download size first:
Once you've got this down to a reasonable size then you can think about the above. You want to download, the smallest amount of content upfront, so in the master. This will improve performance because then the client can cache it. Caching is a good thing, this stops the client having to request the js and css every time they visit a page on your site.
You might also want to think about applying HTTP expiry headers.
Yahoo do a good site on lots of these ideas: http://developer.yahoo.com/performance/rules.html
Also don't put your js in the viewbag. This is unnecessary overhead and load on the server. Just add a reference in your pages!
MVC4 now supports bundling