What is the best/cleanest way to implement A-B testing in asp.net mvc?

Question

What is the best and cleanest way to implement A-B testing in asp.net mvc? That is, when we make new changes to an asp.net mvc web site, we want to test the new html/css/js wit

Answer 1

If you are using the spark view engine, you could probably do it with a variation of the theme filter (http://sparkviewengine.com/documentation/viewlocations#Extendingfilepatternswithdescriptorfilters). For each new visitor to the site, determine if you want them to see the existing or new version of the site and set a cookie. Wire up a descriptor filter that looks for the presence of the cookie and modify the view location to look in the folder containing the modified views. If an alternative view exists, the Spark engine will automatically render it in place of the "normal" view, otherwise it will render the normal view.

If you are using the normal WFVE, then the simplest way to manage this would be to define a folder under Views where your view alternatives live. When you want to provide an alternative view, you place it in a location that matches its position within the normal Views folder but rooted at the alternatives folder e.g. to provide an alternative to Views/Users/login.aspx place your new view at Views/Alternative/Users/login.aspx.

With a convention in place for locating your alternative views, you can extend the WebFormViewEngine and overload CreatePartialView / CreateView to inspect some aspect of the ControllerContext to determine whether to render the default or overloaded view and alter the path as appropriate e.g. changing .../Views/Users/login.aspx to .../Views/Alternative/Users/login.aspx.

Answer 2

Check out FairlyCertain (http://www.fairtutor.com/fairlycertain/) when you get a chance. It's a .NET A/B library that you can pretty much just drop into your project and start writing tests.

Unlike the Javascript libraries from Google and VisualWebsiteOptimizer, everything happens on the server so you don't suffer any performance, user experience or SEO issues. I've been using it in my stuff for a while now and it works quite well.

Answer 3

I suggest you use Display Modes to achieve A/B testing.

But Display Modes just support simple problems by default.

If you already implement Display Modes in some other scenario. You can consider DisplayModeMatrix (just google it). It helps you use Display Modes more efficiency.

https://www.nuget.org/packages/DisplayModeMatrix/

Wth Display Modes you can simply delete/rename views after A/B testing to clean up your project.

Answer 4

Google Content Experiments? It's a Javascript-based solution that doesn't require anything from your backend.

1. You include Google's Javascript on your page
2. The script randomly substitutes elements on your page as defined by your A/B test
3. Google's site shows you a nice breakdown of the results...

Answer 5

There is an A/B testing framework specifically for ASP.NET MVC. This is an open source software I wrote myself when, just like you, didn't find a free tool which works nicely with ASP.NET MVC and doesn't require much setup.

Answer 6

I think there isn't a ready to use solution for this and you will have to improvise.

Try to override your current functionality in well defined points without breaking it. Explicitly draw a border where your regular code and A-B testing code lives.

Inversion of control principle might help a lot here too (i.e. - controller factory could provide derived controller instead of original one). For views&partialviews - you could change viewengine so it would try to look for 'MyPartialViewAB.ascx' instead of 'MyPartialView.ascx'.

And it might be a good idea to take a look what performance counters are (in case you haven't).