Experiment Versions

No matter where the experiment is embedded, each experiment version (or variant) will have a nested set of attributes.

Attributes:

NameTypeDescription
pullsintegerHow many times this version has been selected by the experiment. Subject to when the experiment's lookback window (lookback_days) and when the experiment was last reset (reset_at).
conversionsintegerHow many of the pulls resulted in a conversion. Subject to when the experiment's conversion window (conversion_hours) and when the experiment was last reset (reset_at).
frequentist_weightintegerThis version's weight relative to all other versions. If you have two versions and one has a weight of 1 and one has a weight of 2, the second version will be shown twice as often as the first.

If you prefer to think in terms of percentages, simply ensure that all versions weight add to 100. This is not enforced on the backend, because the system just cares about relative version frequency. Versions with the same weight will have equal likelihood of being shown.

Note that this attribute only impacts the experiment if the experiment's methodology is frequentist. If the experiment is bayesian, this setting will have no impact on which version is selected by the engine.

This value can never be less than 1. Default value: 1.