Creating an experiment
Proba.ai - service for A/B testing in mobile apps. The service implemented classical testing methods, applied Bayesian statistics and Thompson's algorithm.

To get started you need to sign up and add your app.

Creating an experiment
To create an experiment, go to the experiments tab, add an experiment and select the app in the drop-down window.
For the correct experiment operation, it is necessary to set a unique identifier (key) of the experiment, depending on the value of which the logic in the application interface changes. For example, buttonColor.
The key value can be a variable, text, or other type (e.g., JSON, Markdown). It is assumed that the manager and the developer agree on the format of the key value on the user’s side. The service transmits it unchanged. For each experiment, a unique key value is used.

The service allows working with different user segments: new and old. To start, choose a segment relevant to the experiment.
There are two ways to distribute users: manual and automatic.
In the classical (manual) distribution users are divided into groups, the traffic distribution is set manually in percentages, each group is shown a variant.
An experiment run with a manual user distribution can be edited. For example, this may be necessary if one of the experimental variants has low conversion rates. A new version is created when editing the experiment.

The choice of auto user distribution in the experiment is only available when analytical systems are integrated.
To set up automatic distribution, you need to select an analytical system from which the service will receive data and the base metric for traffic optimization.
To facilitate further work on the experiment, an experiment information field can be filled in.
After all fields are filled, the experiment can be started or saved without starting. The experiment can be paused or completed manually.

Note that an experiment that has already been completed cannot be restarted, nor can an experiment that has already started be removed.