How to implement a limited feature rollout (language agnostic) to your users?

Question

I would like to know some common or best practices of rolling out a new website feature to a select group of the userbase.

The users could be, for example, based solel

Answer 1

Google website optimizer seems like exactly what you are looking for.

Answer 2

For a random selection process I like the concept of asking each customer when they have a successful login if they want to participate in beta testing, once the total required or wanted quantity of users is reached you stop asking. In the database I tend to store what server to redirect the user to and run a standard script that moves each user to the correct location upon login.

We design our app development months in advance and avoid changes to the existing schema. The reason is very obvious, of course this is not always possible so when we have a change like this we always fully document the change when it is written and plan the migration for this field as early as possible. This way we have a battle plan of what changes are being made and we can put in place the best solution possible for us. This unfortunately does change depending on the circumstances.

We always run multiple environments, we have production, development and beta generally. This means we 1 do not mess with production services that equal money, we don't have people breaking code and pulling the service offline when optimizing.

Development uses GIT for version monitoring and users never see this as we get all sorts of weird and wonderful experiments uploaded for playing with. It also uses it's own database vs the live data.

With beta, we do migrate specific user data generally but recently we have had a better experience with duplicating the entire database and planning a specific date for the start of beta, what this does is allows users to opt out of beta and others to opt in with minimal changes required to support this option. What we generally do is migrate new data between the 2 databases once per day, new opt-ins and opt-outs only take effect from the time the data has been migrated to the other platform.

We have also had success on a small scale using the existing production database for beta testing some new functions that operated out of their own table so depending on what you're doing data wise using the same live database could be a good option.

I hope this is useful for you... good luck with your testing mate.

Answer 3

My recommendation would be that, for the people who are getting the new feature, the site they are on should be as close as possible to the site that everyone will be on once the feature is public.

In other words, I wouldn't have, for example, conditional logic on a page to determine whether a button should be visible or not, if that condition would only exist during this beta period. Rather, I would determine at login time whether this person is a beta participant or not (that determination might be random, might reference their role, etc). If they are a participant, they would be redirected to a separately deployed site that is deployed from a version control branch.

Once the rollout is complete, the branch is merged in and everyone sees the new feature.

This way, you don't have to worry that the codebase the public beta users are seeing is different (in some way that could break something) from the eventual release codebase.

Answer 4

At my last job we accomplished this using the load balancer and a current revision cookie.

The load balancer set a cookie identifying the revision number of the instance that the user was using. If that cookie was already present, the load balancer would just send that request to a running instance with the corresponding revision. When we deployed a new revision, the load balancer continued to send traffic with an existing revision cookie to their original revision until that revision was no longer running or the user closed their browser. New traffic would get sent to the newly deployed revision. This allowed us to test out changes for a bit while our existing users were continuing to run on the old revision. We could also manually set that cookie to test out the new rev internally on the production environment before turning new traffic onto it. When we were comfortable that the new revision had no major problems we could bring down the old revision and all traffic would start going to the latest revision.

That setup does not support backwards incompatible database changes though. There's pretty much no way to do it where you can have part of your users on one db schema and part on another, and be able to take writes to both and then somehow merge those writes together after you've decided the new rev is ok. I mean, it's possible but it really depends on what the schema changes are exactly and how they effect your app, so you can't do it in a reliable, agnostic way on deployment. For us, we just generally tried not to do backwards incompatible schema changes. If we really did need to, we'd try to postpone the destructive part (dropping a column or table) to a later revision where we could migrate the schema and have both versions running with no adverse effect on current users.