How to define an experiment for first time users in Firebase?

Question

I am trying to create an experiment using Firebase remote config. The criteria is as follows: It should target only new users who have not used the app ( opening the ap

Answer 1

Didn't tried, but it should work. Create an Audience in Firebase Console with following condition: First Open Time is greater than or equal to some date. Where the date should be the next day after you starting an experiment.

Answer 2

I will share our experience with experiments for new users and how the different combinations of targeting work because we worked with it a lot already. You can skip the investigation part if you're not interested and just check the solution that we use now.

Our Investigations Part: Firstly, when we integrated Firebase, we were afraid that Audiences won't work properly for experiments targeting cuz all current users will be treated as new ones after integration, so we checked several approaches and went with an approach of creating specific User Properties that we specified on the client side differently for old/new users. For instance, we created a User Property called adv_experiment_enrolled and specified on client side values 'enrolled'/'not_enrolled', so all new users after installation of this version became 'enrolled' and old ones after updating the version just became 'not_enrolled'. And we just used that User Property as targeting in an experiment. That worked well, but it wasn't a general approach that we could use easily for all experiments and we needed to create User Properties for each new experiment.

So we've tried the Audiences approach after few months of integration that was what @jackes described here with First Open Time user property: https://stackoverflow.com/a/50075684/2723437 And we've got several problems, first of all, seems they had some troubles with populating of that kind of audience and just ~3-5% of new users were getting there. We also created an Audience depending on First Open event itself and used it too, it was populating better and was close to the real number of installs that we had. But we've noticed problems with this approach as well and the biggest one was that an experiment had only 20-30% of users from that Audience. We tested it and noticed by some of our metrics that seems users are not enrolled in this experiment in their first session cuz 1) Firebase takes some time to enroll a user in the Audience and 2) Remote Config has 12h cache by default, so it wasn't really a data for most of the new installations.

A solution that seems to work well for now:
We were surprised with that Firebase does have User Property for First Open Time, but doesn't allow to use this as the targeting for experiments (It would be very helpful to solve this problem tbh), so we just decided to try our good experience with User Property targeting and apply the general approach First Open Time user property, so we've created our own custom_first_open_time especially to target installations after some specific time (we just used current timestamps for platforms in seconds).
Important notes:
- You have to set up user properties before loading a remote config.
- You have to keep this first open time on the client side persistently once you generated it (usually you use NSUserDefaults/SharedPreferences for iOS/Android for that)

Sample of experiment configuration: