ASP.NET MVC and EF Code First Memory Usage

Question

I have an application built in ASP.NET MVC 3 that uses SQL CE for storage and EF CTP 5 for data access.

I\'ve deployed this site to a shared host only to find that i

Answer 1

The funny thing is, I think they got their estimate from this URL:

http://blog.whitesites.com/w3wp-exe-using-too-much-memory-and-resources__633900106668026886_blog.htm

P.S. It's great article to check and see if you're doing anything that the guy is describing. (For example caching your pages)

P.S.S. Just checked our system and it's running at 50 megs currently. We're using MVC 2 and EF CTP 4.

Answer 2

We did manage to reduce our memory footprint quite significantly. The IIS worker process now sits around 50mb compared to the 100+mb before.

Below are some of the things that helped us:

• Check the basics. Make sure you compile in release mode and set compilation debug to false in web.config. It's easy to forget such things.
• Use DEBUG symbols for diagnostic code. An example of this would be when using tools like NHProf (yes I've been caught out by this before). The easiest thing is to wrap such code in an #if DEBUG directive to ensure it's not compiled into the release of your application.
• Don't forget about SQL. ORMs make it too easy to ignore how your application is talking to your database. Using SQL Profiler or tools like EFProf/NHProf can show you exactly what is going on. In the case of EF you will probably feel a little ill afterwards, especially if you make significant use of lazy loading. Once you've got over this, you can begin to optimize (see point below).
• Lazy loading is convenient but shouldn't be used in MVC views (IMO). This was one of the root causes of our high memory usage. The home page of our web site was creating 59 individual queries due to lazy loading (SELECT N+1). After creating a specific viewmodel for this page and eagerly loading the associations we needed we got down to 6 queries that executed in half the time.
• Design patterns are there to guide you, not rule the development of your application. I tend to follow a DDD approach where possible. In this case I didn't really want to expose foreign keys on my domain model. However since EF does not handle many-to-one associations quite as well as NH (it will issue another query just to get the foreign key of an object we already have in memory), I ended up with an additional query (per object) displayed on my page. In this case I decided that I could put up with a bit of code smell (including the FK in my model) for the sake of improved performance.
• A common "solution" is to throw caching at performance issues. It's important to identify the real problem before you formulate your caching strategy. I could have just applied output caching to our home page (see note below) but this doesn't change the fact that I have 59 queries hitting my database when the cache expires.

A note on output caching: When ASP.NET MVC was first released we were able to do donut caching, that is, caching a page APART from a specific region(s). The fact that this is no longer possible makes output caching pretty useless if you have user specific information on the page. For example, we have a login status within the navigation menu of the site. This alone means I can't use Output caching for the page as it would also cache the login status.

Ultimately there is no hard and fast rule on how to optimize an application. The biggest improvement in our application's performance came when we stopped using the ORM for building our associations (for the public facing part of our site) and instead loaded them manually into our viewmodels. We couldn't use EF to eagerly load them as there were too many associations (resulting in a messy UNION query).

An example was our tagging mechanism. Entities like BlogPost and Project can be tagged. Tags and tagable entities have a many-to-many relationship. In our case it was better to retrieve all tags and cache them. We then created a linq projection to cache the association keys for our tagable entities (e.g. ProjectId / TagId). When creating the viewmodel for our page we could then build up the tags for each tagable entity without hitting the database. Again, this was specific to our application but it yielded a massive improvement in performance and in lowering our memory usage.

Some of the resources / tools we used along the way:

• EFProf - to monitor the queries generated by Entity Framework (free trial available)
• ANTS Memory Profiler (free trial available)
• Windows performance monitor (perfmon)
• Tess Ferrandez's blog
• Lots of coffee :)

Whilst we did make improvements that would take us under the hosting company's (Arvixe) application pool limits, I do feel a sense of duty to advise people who are looking at their Windows Reseller plans, that such restrictions are in place (since Arvixe do not mention this anywhere when advertising the plan). So when something looks too good to be true (unlimited x,y,z), it usually is.