summaryrefslogtreecommitdiffstats
path: root/browser/components/newtab/content-src/asrouter/docs/experiment-guide.md
diff options
context:
space:
mode:
Diffstat (limited to 'browser/components/newtab/content-src/asrouter/docs/experiment-guide.md')
-rw-r--r--browser/components/newtab/content-src/asrouter/docs/experiment-guide.md52
1 files changed, 52 insertions, 0 deletions
diff --git a/browser/components/newtab/content-src/asrouter/docs/experiment-guide.md b/browser/components/newtab/content-src/asrouter/docs/experiment-guide.md
new file mode 100644
index 0000000000..ac2784bb1f
--- /dev/null
+++ b/browser/components/newtab/content-src/asrouter/docs/experiment-guide.md
@@ -0,0 +1,52 @@
+# How to run experiments with ASRouter
+
+This guide will tell you how to run an experiment with ASRouter messages.
+Note that the actual experiment process and infrastructure is handled by
+the experiments team (#ask-experimenter).
+
+## Why run an experiment
+
+* To measure the effect of a message on a Firefox metric (e.g. retention)
+* To test a potentially risky message on a smaller group of users
+* To compare the performance of multiple variants of messages in a controlled way
+
+## Choose cohort IDs and request an experiment
+
+First you should decide on a cohort ID (this can be any arbitrary unique string) for each
+individual group you need to segment for your experiment.
+
+For example, if I want to test two variants of an FXA Snippet, I might have two cohort IDs,
+`FXA_SNIPPET_V1` and `FXA_SNIPPET_V2`.
+
+You will then [request](https://experimenter.services.mozilla.com/) a new "pref-flip" study with the Firefox Experiments team.
+The preferences you will submit will be based on the cohort IDs you chose.
+
+For the FXA Snippet example, your preference name would be `browser.newtabpage.activity-stream.asrouter.providers.snippets` and values would be:
+
+Control (default value)
+```json
+{"id":"snippets","enabled":true,"type":"remote","url":"https://snippets.cdn.mozilla.net/%STARTPAGE_VERSION%/%NAME%/%VERSION%/%APPBUILDID%/%BUILD_TARGET%/%LOCALE%/release/%OS_VERSION%/%DISTRIBUTION%/%DISTRIBUTION_VERSION%/","updateCycleInMs":14400000}
+```
+
+Variant 1:
+```json
+{"id":"snippets", "cohort": "FXA_SNIPPET_V1", "enabled":true,"type":"remote","url":"https://snippets.cdn.mozilla.net/%STARTPAGE_VERSION%/%NAME%/%VERSION%/%APPBUILDID%/%BUILD_TARGET%/%LOCALE%/release/%OS_VERSION%/%DISTRIBUTION%/%DISTRIBUTION_VERSION%/","updateCycleInMs":14400000}
+```
+
+Variant 2:
+```json
+{"id":"snippets", "cohort": "FXA_SNIPPET_V1", "enabled":true,"type":"remote","url":"https://snippets.cdn.mozilla.net/%STARTPAGE_VERSION%/%NAME%/%VERSION%/%APPBUILDID%/%BUILD_TARGET%/%LOCALE%/release/%OS_VERSION%/%DISTRIBUTION%/%DISTRIBUTION_VERSION%/","updateCycleInMs":14400000}
+```
+
+## Add targeting to your messages
+
+You must now check for the cohort ID in the `targeting` expression of the messages you want to include in your experiments.
+
+For the previous example, you wold include the following to target the first cohort:
+
+```json
+{
+ "targeting": "providerCohorts.snippets == \"FXA_SNIPPET_V1\""
+}
+
+```