I configured Firebase Remote Config A/B testing for Android, and we did rollout on at least 10K devices. For some reason, I see "0 users" in my A/B test after more than 24 hours.
Firebase GMS version is: 11.8.0
Should it show A/B participants in real-time or it's ok to see 0 users after 24 hours?
P.S: We are able to get AB test variants on test devices through Firebase Instance Id, it works well.
The simplest experiment which is running has only app package as a target, with no additional filters. And it shows 0 users as well.
Finally, we found an answer!
Maybe somebody will find it helpful:
For now, it happens (no data in Firebase remote config A/B test experiment) if you have an activation event configured for A/B test experiment.
If you have 2 different experiments, both will fail to get results even if you have "activation event" configured only in 1 of them. Additionally, remote config will not work as well, you'll be able to get only default values.
We already reported to Google about, so they'll fix it at some point I hope.
Another useful info which is really hard to get:
- How long is it ok to see "0 Total Users" in experiment I've just started?
It takes many hours before you can see any data in your experiment. We were able to see results only after 21 hours after experiment start, so if you configured everything well, don't worry and wait for at least 24 hours. It will show 0 "Total Users" for many hours after the start.
- Should I use app versionName or versionCode in "Version" field of experiment setup?
You should use versionName.
Some useful info from support:
- Firebase SDK
Make sure your users have the version of your app with the latest SDK.
- Since your experiment is with Remote Config
When activateFetched() is called, all events from that point on will be tagged with the experiment. If you have a goal or activation event that happens before activateFetched(), such as automatic events like first_open, session_start, etc., the experiment setup might be wrong.
- Are you using an Activation Event?
Make sure to call fetch() and activateFetched() before the activation event occurs.
- Experiment ID of the experiments (if support asks you about)
It's the number at the end of the URL while viewing experiment results.
This debugging log could be useful to get what is going on
Also:
The good way to check if your experiment is working now is to set it to a specific version you didn't publish yet and check logs from remote config with the fresh app install(or erase all app data & restart).
It should show different variant every time you reinstall the app, since your Firebase Instance ID changes after app reinstall/app data erase. If you see variants change - then A/B test is running well.
In your "build.graddle": don't forget to set the same versionName which you set in experiment setup.
来源：https://stackoverflow.com/questions/49311141/firebase-remote-config-a-b-testing-shows-no-results-after-24-hours