diff options
Diffstat (limited to 'browser/components/newtab/docs/v2-system-addon')
9 files changed, 705 insertions, 0 deletions
diff --git a/browser/components/newtab/docs/v2-system-addon/about_home_startup_cache.md b/browser/components/newtab/docs/v2-system-addon/about_home_startup_cache.md new file mode 100644 index 0000000000..0366bd3e29 --- /dev/null +++ b/browser/components/newtab/docs/v2-system-addon/about_home_startup_cache.md @@ -0,0 +1,86 @@ +# The `about:home` startup cache + +By default, a user's browser session starts with a single window and a single tab, pointed at about:home. This means that it's important to ensure that `about:home` loads as quickly as possible to provide a fast overall startup experience. + +`about:home`, which is functionally identical to `about:newtab`, is generated dynamically by calculating an appropriate state object in the parent process, and passing it down to a content process into the React library in order to render the final interactive page. This is problematic during the startup sequence, as calculating that initial state can be computationally expensive, and requires multiple reads from the disk. + +The `about:home` startup cache is an attempt to address this expense. It works by assuming that between browser sessions, `about:home` _usually_ doesn't need to change. + +## Components of the `about:home` startup cache mechanism + +There are 3 primary components to the cache mechanism: + +### The HTTP Cache + +The HTTP cache is normally used for caching webpages retrieved over the network, but seemed like the right fit for storage of the `about:home` cache as well. + +The HTTP cache is usually queried by the networking stack when browsing the web. The HTTP cache is, however, not typically queried when accessing `chrome://` or `resource://` URLs, so we have to do it ourselves, manually for the `about:home` case. This means giving `about:home` special capabilities for populating and reading from the HTTP cache. In order to avoid potential security issues, this requires that we sequester `about:home` / `about:newtab` in their own special content process. The "privileged about content process" exists for this purpose, and is also used for `about:logins` and `about:certificate`. + +The HTTP cache lives in the parent process, and so any read and write operations need to be initiated in the parent process. Thankfully, however, the HTTP cache accepts data using `nsIOutputStream` and serves it using `nsIInputStream`. We can send `nsIInputStream` over the message manager, and convert an `nsIInputStream` into an `nsIOutputStream`, so we have everything we need to efficiently communicate with the "privileged about content process" to save and retrieve page data. + +The official documentation for the HTTP cache [can be found here](https://firefox-source-docs.mozilla.org/networking/cache2/doc.html). + +### `AboutHomeStartupCache` + +This singleton component lives inside of `BrowserGlue` to avoid having to load yet another JSM out of the `omni.ja` file in the parent process during startup. + +`AboutHomeStartupCache` is responsible for feeding the "privileged about content process" with the `nsIInputStream`'s that it needs to present the initial `about:home` document. It is also responsible for populating the cache with updated versions of `about:home` that are sent by the "privileged about content process". + +Since accessing the HTTP cache is asynchronous, there is an opportunity for a race, where the cache can either be accessed and available before the initial `about:home` is requested, or after. To accommodate for both cases, the `AboutHomeStartupCache` constructs `nsIPipe` instances, which it sends down to the "privileged about content process" as soon as one launches. + +If the HTTP cache entry is already available when the process launches, and cached data is available, we connect the cache to the `nsIPipe`'s to stream the data down to the "privileged about content process". + +If the HTTP cache is not yet available, we hold references to those `nsIPipe` instances, and wait until the cache entry is available. Only then do we connect the cache entry to the `nsIPipe` instances to send the data down to the "privileged about content process". + +### `AboutNewTabService` + +The `AboutNewTabService` is used by the `AboutRedirector` in both the parent and content processes to determine how to handle attempts to load `about:home` and `about:newtab`. + +There are distinct versions of the `AboutNewTabService` - one for the parent process (`BaseAboutNewTabService`), and one for content processes (`AboutNewTabChildService`, which inherits from `BaseAboutNewTabService`). + +The `AboutRedirector`, when running inside of a "privileged about content process" knows to direct attempts to load `about:home` to `AboutNewTabChildService`'s `aboutHomeCacheChannel` method. This method is then responsible for choosing whether or not to return an `nsIChannel` for the cached document, or for the dynamically generated version of `about:home`. + +### `AboutHomeStartupCacheChild` + +This singleton component lives inside of the "privileged about content process", and is initialized as soon as the message is received from the parent that includes the `nsIInputStream`'s that will be used to potentially load from the cache. + +When the `AboutRedirector` in the "privileged about content process" notices that a request has been made to `about:home`, it asks `nsIAboutNewTabService` to return a new `nsIChannel` for that document. The `AboutNewTabChildService` then checks to see if the `AboutHomeStartupCacheChild` can return an `nsIChannel` for any cached content. + +If, at this point, nothing has been streamed from the parent, we fall back to loading the dynamic `about:home` document. This might occur if the cache doesn't exist yet, or if we were too slow to pull it off of the disk. Subsequent attempts to load `about:home` will bypass the cache and load the dynamic document instead. This is true even if the privileged about content process crashes and a new one is created. + +The `AboutHomeStartupCacheChild` will also be responsible for generating the cache periodically. Periodically, the `AboutNewTabService` will send down the most up-to-date state for `about:home` from the parent process, and then the `AboutHomeStartupCacheChild` will generate document markup using ReactDOMServer within a `ChromeWorker`. After that's generated, the "privileged about content process" will send up `nsIInputStream` instances for both the markup and the script for the initial page state. The `AboutHomeStartupCache` singleton inside of `BrowserGlue` is responsible for receiving those `nsIInputStream`'s and persisting them in the HTTP cache for the next start. + +## What is cached? + +Two things are cached: + +1. The raw HTML mark-up of `about:home`. +2. A small chunk of JavaScript that "hydrates" the markup through the React libraries, allowing the page to become interactive after painting. + +The JavaScript being cached cannot be put directly into the HTML mark-up as inline script due to the CSP of `about:home`, which does not allow inline scripting. Instead, we load a script from `about:home?jscache`. This goes through the same mechanism for retrieving the HTML document from the cache, but instead pulls down the cached script. + +If the HTML mark-up is cached, then we presume that the script is also cached. We cannot cache one and not the other. If only one cache exists, or only one has been sent down to the "privileged about content process" by the time the `about:home` document is requested, then we fallback to loading the dynamic `about:home` document. + +## Refreshing the cache + +The cache is refreshed periodically by having `ActivityStreamMessageChannel` tell `AboutHomeStartupCache` when it has sent any messages down to the preloaded `about:newtab`. In general, such messages are a good hint that something visual has updated for the next `about:newtab`, and that the cache should probably be refreshed. + +`AboutHomeStartupCache` debounces notifications about such messages, since they tend to be bursty. + +## Invalidating the cache + +It's possible that the composition or layout of `about:home` will change over time from release to release. When this occurs, it might be desirable to invalidate any pre-existing cache that might exist for a user, so that they don't see an outdated `about:home` on startup. + +To do this, we set a version number on the cache entry, and ensure that the version number is equal to our expectations on startup. If the version number does not match our expectation, then the cache is discarded and the `about:home` document will be rendered dynamically. + +The version number is currently set to the application build ID. This means that when the application updates, the cache is invalidated on the first restart after a browser update is applied. + +## Handling errors + +`about:home` is typically the first thing that the user sees upon starting the browser. It is critically important that it function quickly and correctly. If anything happens to go wrong when retrieving or saving to the cache, we should fall back to generating the document dynamically. + +As an example, it's theoretically possible for the browser to crash while in the midst of saving to the cache. In that case, we might have a partial document saved, or a partial script saved - neither of which is acceptable. + +Thankfully, the HTTP cache was designed with resilience in mind, so partially written entries are automatically discarded, which allows us to fall back to the dynamic page generation mode. + +As additional redundancy to that resilience, we also make sure to create a new nsICacheEntry every time the cache is populated, and write the version metadata as the last step. Since the version metadata is written last, we know that if it's missing when we try to load the cache that the writing of the page and the script did not complete, and that we should fall back to dynamically rendering the page. diff --git a/browser/components/newtab/docs/v2-system-addon/data_events.md b/browser/components/newtab/docs/v2-system-addon/data_events.md new file mode 100644 index 0000000000..78236bc3b1 --- /dev/null +++ b/browser/components/newtab/docs/v2-system-addon/data_events.md @@ -0,0 +1,19 @@ +# Metrics we collect + +By default, the about:newtab, about:welcome and about:home pages in Firefox (the pages you see when you open a new tab and when you start the browser), will send data back to Mozilla servers about usage of these pages. The intent is to collect data in order to improve the user's experience while using Activity Stream. Data about your specific browsing behaior or the sites you visit is **never transmitted to any Mozilla server**. At any time, it is easy to **turn off** this data collection by [opting out of Firefox telemetry](https://support.mozilla.org/kb/share-telemetry-data-mozilla-help-improve-firefox). + +Data is sent to our servers in the form of discrete HTTPS 'pings' or messages whenever you do some action on the Activity Stream about:home, about:newtab or about:welcome pages. We try to minimize the amount and frequency of pings by batching them together. + +At Mozilla, [we take your privacy very seriously](https://www.mozilla.org/privacy/). The Activity Stream page will never send any data that could personally identify you. We do not transmit what you are browsing, searches you perform or any private settings. Activity Stream does not set or send cookies, and uses [Transport Layer Security](https://en.wikipedia.org/wiki/Transport_Layer_Security) to securely transmit data to Mozilla servers. + +The data collected in the Activity Stream is documented +(along with the other data collected in Firefox Desktop) +in the [Glean Dictionary](https://dictionary.telemetry.mozilla.org/apps/firefox_desktop). + +Ping specifically collected on Firefox Home (New Tab) include: +* ["newtab"](https://dictionary.telemetry.mozilla.org/apps/firefox_desktop/pings/newtab) +* ["pocket-button"](https://dictionary.telemetry.mozilla.org/apps/firefox_desktop/pings/pocket-button) +* ["top-sites"](https://dictionary.telemetry.mozilla.org/apps/firefox_desktop/pings/top-sites) +* ["quick-suggest"](https://dictionary.telemetry.mozilla.org/apps/firefox_desktop/pings/quick-suggest) +* ["messaging-system"](https://dictionary.telemetry.mozilla.org/apps/firefox_desktop/pings/messaging-system) +* ["spoc"](https://dictionary.telemetry.mozilla.org/apps/firefox_desktop/pings/spoc) diff --git a/browser/components/newtab/docs/v2-system-addon/geo_locale.md b/browser/components/newtab/docs/v2-system-addon/geo_locale.md new file mode 100644 index 0000000000..4641e5d001 --- /dev/null +++ b/browser/components/newtab/docs/v2-system-addon/geo_locale.md @@ -0,0 +1,23 @@ +# Custom `geo`, `locale`, and update channels + +There are instances where you may need to change your local build's locale, geo, and update channel (such as changes to the visibility of Discovery Stream on a per-geo/locale basis in `ActivityStream.sys.mjs`). + +## Changing update channel + +- Change `app.update.channel` to desired value (eg: `release`) by editing `LOCAL_BUILD/Contents/Resources/defaults/pref/channel-prefs.js`. (**NOTE:** Changing pref `app.update.channel` from `about:config` seems to have no effect!) + +## Changing geo + +- Set `browser.search.region` to desired geo (eg `CA`) + +## Changing locale + +*Note: These prefs are only configurable on a nightly or local build.* + +- Toggle `extensions.langpacks.signatures.required` to `false` +- Toggle `xpinstall.signatures.required` to `false` +- Toggle `intl.multilingual.downloadEnabled` to `true` +- Toggle `intl.multilingual.enabled` to `true` +- For Mac and Linux builds, open the [langpack](https://archive.mozilla.org/pub/firefox/nightly/latest-mozilla-central-l10n/linux-x86_64/xpi/) for target locale in your local build (eg `firefox-70.0a1.en-CA.langpack.xpi` if you want an `en-CA` locale). +- For Windows, use [https://archive.mozilla.org/pub/firefox/nightly/latest-mozilla-central-l10n/](https://archive.mozilla.org/pub/firefox/nightly/latest-mozilla-central-l10n/) +- In `about:preferences` click "Set Alternatives" under "Language", move desired locale to the top position, click OK, click "Apply And Restart" diff --git a/browser/components/newtab/docs/v2-system-addon/mochitests.md b/browser/components/newtab/docs/v2-system-addon/mochitests.md new file mode 100644 index 0000000000..da77874401 --- /dev/null +++ b/browser/components/newtab/docs/v2-system-addon/mochitests.md @@ -0,0 +1,26 @@ +# Mochitests + +We use [mochitests](https://firefox-source-docs.mozilla.org/testing/browser-chrome/) to do functional (and possibly integration) testing. Mochitests are part of Firefox and allow us to test activity stream literally as you would use it. + +Mochitests live in `test/browser`, and as of this writing, they are all the `browser-chrome` flavor of mochitests. They currently only run against the bootstrapped version of the add-on in system-addon, not the test pilot version at the top level directory. + +## Adding New Tests + +If you add new tests, make sure to list them in the `browser.ini` file. You will see the other tests there. Add a new entry with the same format as the others. You can also add new JS or HTML files by listing in under `support-files`. Make sure to start your test name with "browser_", so that the test suite knows the pick it up. E.g: "browser_as_my_new_test.js". + +## Writing Tests + +Here are a few tips for writing mochitests: + +* Only write mochitests for testing the interaction of multiple components on the page and to make sure that the protocol is working. +* If you need to access the content page, use `ContentTask.spawn`: + +```js +ContentTask.spawn(gBrowser.selectedBrowser, null, function* () { + content.wrappedJSObject.foo(); +}); +``` + +The above calls the function `foo` that exists in the page itself. You can also access the DOM this way: `content.document.querySelector`, if you want to click a button or do other things. You can even you use assertions inside this callback to check DOM state. + +* Nobody likes to see intermittent oranges in their tests, so read the [docs on how to avoid them](https://firefox-source-docs.mozilla.org/testing/intermittent/)! diff --git a/browser/components/newtab/docs/v2-system-addon/preferences.md b/browser/components/newtab/docs/v2-system-addon/preferences.md new file mode 100644 index 0000000000..ec6ba82491 --- /dev/null +++ b/browser/components/newtab/docs/v2-system-addon/preferences.md @@ -0,0 +1,270 @@ +# Preferences + +## Preference branch + +The preference branch for activity stream is `browser.newtabpage.activity-stream.`. +Any preferences defined in the preference configuration will be relative to that +branch. For example, if a preference is defined with the name `foo`, the full +preference as it is displayed in `about:config` will be `browser.newtabpage.activity-stream.foo`. + +## Defining new preferences + +All preferences for Activity Stream should be defined in the `PREFS_CONFIG` Array +found in `lib/ActivityStream.sys.mjs`. +The configuration object should have a `name` (the name of the pref), a `title` +that describes the functionality of the pref, and a `value`, the default value +of the pref. Optionally a `getValue` function can be provided to dynamically +generate a default pref value based on args, e.g., geo and locale. For +developers-specific defaults, an optional `value_local_dev` will be used instead +of `value`. For example: + +```js +{ + name: "telemetry.log", + title: "Log telemetry events in the console", + value: false, + value_local_dev: true, + getValue: ({geo}) => geo === "CA" +} +``` + +### IMPORTANT: Setting test-specific values for Mozilla Central + +If a feed or feature behind a pref makes any network calls or would other be +disruptive for automated tests and that pref is on by default, make sure you +disable it for tests in Mozilla Central. + +You should create a bug in Bugzilla and a patch that adds lines to turn off your +pref in the following files: + +- layout/tools/reftest/reftest-preferences.js +- testing/profiles/prefs_general.js +- testing/talos/talos/config.py + +You can see an example in [this patch](https://github.com/mozilla/activity-stream/pull/2977). + +## Reading, setting, and observing preferences from `.jsm`s + +To read/set/observe Activity Stream preferences, construct a `Prefs` instance found in `lib/ActivityStreamPrefs.sys.mjs`. + +```js +// Import Prefs +const { Prefs } = ChromeUtils.importESModule( + "resource://activity-stream/lib/ActivityStreamPrefs.sys.mjs" +); + +// Create an instance +const prefs = new Prefs(); +``` + +The `Prefs` utility will set the Activity Stream branch for you by default, so you +don't need to worry about prefixing every pref with `browser.newtabpage.activity-stream.`: + +```js +const prefs = new Prefs(); + +// This will return the value of browser.newtabpage.activity-stream.foo +prefs.get("foo"); + +// This will set the value of browser.newtabpage.activity-stream.foo to true +prefs.set("foo", true); + +// This will call aCallback when browser.newtabpage.activity-stream.foo is changed +prefs.observe("foo", aCallback); + +// This will stop listening to browser.newtabpage.activity-stream.foo +prefs.ignore("foo", aCallback); +``` + +See :searchfox:`toolkit/modules/Preferences.sys.mjs <source/toolkit/modules/Preferences.sys.mjs>` +for more information about what methods are available. + +## Discovery Stream Preferences + +Preferences specific to the Discovery Stream are nested under the sub-branch `browser.newtabpage.activity-stream.discoverystream` (with the exception of `browser.newtabpage.blocked`). + +### `browser.newtabpage.activity-stream.discoverystream.flight.blocks` + +- Type: `string (JSON)` +- Default: `{}` +- Pref Type: AS + +Not intended for user configuration, but is programmatically updated. Used for tracking blocked flight IDs when a user dismisses a SPOC. Keys are flight IDs. Values don't have a specific meaning. + +### `browser.newtabpage.blocked` + +- Type: `string (JSON)` +- Default: `null` +- Pref Type: AS + +Not intended for user configuration, but is programmatically updated. Used for tracking blocked story IDs when a user dismisses one. Keys are story IDs. Values don't have a specific meaning. + +### `browser.newtabpage.activity-stream.discoverystream.config` + +- Type `string (JSON)` +- Default: + ```json + { + "api_key_pref": "extensions.pocket.oAuthConsumerKey", + "collapsible": true, + "enabled": true, + "personalized": true, + } + ``` + - `api_key_pref` (string): The name of a variable containing the key for the Pocket API. + - `collapsible` (boolean): Controls whether the sections in new tab can be collapsed. + - `enabled` (boolean): Controls whether DS is turned on and is programmatically set based on a user's locale. DS enablement is a logical `AND` of this and the value of `browser.newtabpage.activity-stream.discoverystream.enabled`. + - `personalized` (boolean): When this is `true` personalized content based on browsing history will be displayed. + - `unused_key` (string): This is not set by default and is unused by this codebase. It's a standardized way to differentiate configurations to prevent experiment participants from being unenrolled. + +### `browser.newtabpage.activity-stream.discoverystream.enabled` + +- Type: `boolean` +- Default: `true` +- Pref Type: Firefox + +When this is set to `true` the Discovery Stream experience will show up if `enabled` is also `true` on `browser.newtabpage.activity-stream.discoverystream.config`. Otherwise the old Activity Stream experience will be shown. + +### `browser.newtabpage.activity-stream.discoverystream.endpointSpocsClear` + +- Type: `string (URL)` +- Default: `https://spocs.getpocket.com/user` +- Pref Type: AS + +Endpoint for when a user opts-out of sponsored content to delete the corresponding data from the ad server. + +### `browser.newtabpage.activity-stream.discoverystream.endpoints` + +- Type: `string (URLs, CSV)` +- Default: `https://getpocket.cdn.mozilla.net/,https://spocs.getpocket.com/` +- Pref Type: AS + +A list of endpoints that are allowed to be used by Discovery Stream for remote content (eg: story metadata) and configuration (eg: remote layout definitions for experimentation). + +### `browser.newtabpage.activity-stream.discoverystream.hardcoded-basic-layout` + +- Type: `boolean` +- Default: `false` +- Pref Type: Firefox + +If this is `false` the default hardcoded layout is used, and if it's `true` then an alternate hardcoded layout (that currently simulates the older AS experience) is used. + +### `browser.newtabpage.activity-stream.discoverystream.rec.impressions` + +- Type: `string (JSON)` +- Default: `{}` +- Pref Type: AS + +Programmatically generated hash table where the keys are recommendation IDs and the values are timestamps representing the first impression. + +### `browser.newtabpage.activity-stream.discoverystream.spoc.impressions` + +- Type: `string (JSON)` +- Default: `{}` +- Pref Type: AS + +Programmatically generated hash table where the keys are sponsored content IDs and the values are arrays of timestamps for every impression. + +### `browser.newtabpage.activity-stream.discoverystream.locale-list-config` + +- Type: `string (CSV, locales)` +- Default: `null` +- Pref Type: Firefox + +A comma separated list of locales that by default have stories enabled in newtab. It overrides what might be in region-stories-config. So if I set this to "en-US,en-CA,en-GB", all users with a English browser would see newtab stories, even if their region was not in region-stories-config list. + +### `browser.newtabpage.activity-stream.discoverystream.region-stories-config` + +- Type: `string (CSV, regions)` +- Default: `US,DE,CA,GB,IE,CH,AT,BE` +- Pref Type: Firefox + +A comma separated list of geos that by default have stories enabled in newtab. It matches the client's geo with that list, then looks for a matching locale. + +### `browser.newtabpage.activity-stream.discoverystream.region-spocs-config` + +- Type: `string (CSV, regions)` +- Default: `US,CA,DE` +- Pref Type: Firefox + +A comma separated list of geos that by default have spocs enabled in newtab. It matches the client's geo with that list. + +### `browser.newtabpage.activity-stream.discoverystream.region-layout-config` + +- Type: `string (CSV, regions)` +- Default: `US,CA,GB,DE,IE,CH,AT,BE` +- Pref Type: Firefox + +A comma separated list of geos that have 7 rows of stories enabled in newtab. It matches the client's geo with that list. + +### `browser.newtabpage.activity-stream.discoverystream.region-basic-layout` + +- Type: `boolean` +- Default: false +- Pref Type: AS + +If this is `true` newtabs with stories enabled see 1 row. It is set programmatically based on the result from region-layout-config. + +### `browser.newtabpage.activity-stream.discoverystream.spocs-endpoint` + +- Type: `string (URL)` +- Default: `null` +- Pref Type: Firefox + +Override to specify endpoint for SPOCs. Will take precedence over remote and hardcoded layout SPOC endpoints. + +### `browser.newtabpage.activity-stream.discoverystream.personalization.version` + +- Type: `integer` +- Default: `1` +- Pref Type: Firefox + +This controls what version of personalization we should use to score newtab stories. + +### `browser.newtabpage.activity-stream.discoverystream.personalization.modelKeys` + +- Type: `string (CSV)` +- Default: `nb_model_arts_and_entertainment, nb_model_autos_and_vehicles, nb_model_beauty_and_fitness, nb_model_blogging_resources_and_services, nb_model_books_and_literature, nb_model_business_and_industrial, nb_model_computers_and_electronics, nb_model_finance, nb_model_food_and_drink, nb_model_games, nb_model_health, nb_model_hobbies_and_leisure, nb_model_home_and_garden, nb_model_internet_and_telecom, nb_model_jobs_and_education, nb_model_law_and_government, nb_model_online_communities, nb_model_people_and_society, nb_model_pets_and_animals, nb_model_real_estate, nb_model_reference, nb_model_science, nb_model_shopping, nb_model_sports, nb_model_travel` +- Pref Type: Firefox + +This is a configuration for personalization version 2. It is a list of topics the algorithm uses to score stories by. + +### `browser.newtabpage.activity-stream.discoverystream.recs.personalized` + +- Type: `boolean` +- Default: false +- Pref Type: Firefox + +This controls if newtab story personalization includes regular stories or not. See spocs.personalized for sponsored content. + +### `browser.newtabpage.activity-stream.discoverystream.spocs.personalized` + +- Type: `boolean` +- Default: true +- Pref Type: Firefox + +This controls if newtab story personalization includes sponsored content or not. See recs.personalized for regular stories. + +### `browser.newtabpage.activity-stream.discoverystream.isCollectionDismissible` + +- Type: `boolean` +- Default: true +- Pref Type: Firefox + +This controls if newtab story collections are dismissible or not. + +### `browser.newtabpage.activity-stream.feeds.section.topstories` + +- Type: `boolean` +- Default: true +- Pref Type: Firefox + +This controls if the user should see newtab stories or not. It is set by the user via about:preferences#home + +### `browser.newtabpage.activity-stream.feeds.system.topstories` + +- Type: `boolean` +- Default: false +- Pref Type: AS + +Not intended for user configuration, but is programmatically set. It also controls if the user should see newtab stories or not. It is set at run time, and computed based on the locale/region. diff --git a/browser/components/newtab/docs/v2-system-addon/sections.md b/browser/components/newtab/docs/v2-system-addon/sections.md new file mode 100644 index 0000000000..332cf5b26d --- /dev/null +++ b/browser/components/newtab/docs/v2-system-addon/sections.md @@ -0,0 +1,82 @@ +# Sections + +Each section in Activity Stream displays data from a corresponding section feed +in a standardised `Section` UI component. Each section feed is responsible for +listening to events and updating the section options such as the title, icon, +and rows (the cards for the section to display). + +The `Section` UI component displays the rows provided by the section feed. If no +rows are available it displays an empty state consisting of an icon and a +message. Optionally, the section may have a info option menu that is displayed +when users hover over the info icon. + +On load, `SectionsManager` and `SectionsFeed` in `SectionsManager.sys.mjs` add the +sections configured in the `BUILT_IN_SECTIONS` map to the state. These sections +are initially disabled, so aren't visible. The section's feed may use the +methods provided by the `SectionsManager` to enable its section and update its +properties. + +The section configuration in `BUILT_IN_SECTIONS` consists of a generator +function keyed by the pref name for the section feed. The generator function +takes an `options` argument as the only parameter, which is passed the object +stored as serialised JSON in the pref `{feed_pref_name}.options`, or the empty +object if this doesn't exist. The generator returns a section configuration +object which may have the following properties: + +```{eval-rst} ++--------------------+---------------------+-------------------------------------------------------------------------------------------------------------------+ +| Property | Type | Description | ++====================+=====================+===================================================================================================================+ +| id | String | Non-optional unique id. | ++--------------------+---------------------+-------------------------------------------------------------------------------------------------------------------+ +| title | Localisation object | Has property `id`, the string localisation id, and optionally a `values` object to fill in placeholders. | ++--------------------+---------------------+-------------------------------------------------------------------------------------------------------------------+ +| icon | String | Icon id. New icons should be added in icons.scss. | ++--------------------+---------------------+-------------------------------------------------------------------------------------------------------------------+ +| maxRows | Integer | Maximum number of rows of cards to display. Should be >= 1. | ++--------------------+---------------------+-------------------------------------------------------------------------------------------------------------------+ +| contextMenuOptions | Array of strings | The menu options to provide in the card context menus. | ++--------------------+---------------------+-------------------------------------------------------------------------------------------------------------------+ +| shouldHidePref | Boolean | If true, will the section preference in the preferences pane will not be shown. | ++--------------------+---------------------+-------------------------------------------------------------------------------------------------------------------+ +| pref | Object | Configures the section preference to show in the preferences pane. Has properties `titleString` and `descString`. | ++--------------------+---------------------+-------------------------------------------------------------------------------------------------------------------+ +| emptyState | Object | Configures the empty state of the section. Has properties `message` and `icon`. | ++--------------------+---------------------+-------------------------------------------------------------------------------------------------------------------+ +``` + +## Section feeds + +Each section feed should be controlled by the pref `feeds.section.{section_id}`. + +### Enabling the section + +The section feed must listen for the events `INIT` (dispatched when Activity +Stream is initialised) and `FEED_INIT` (dispatched when a feed is re-enabled +having been turned off, with the feed id as the `data`). On these events it must +call `SectionsManager.enableSection(id)`. Care should be taken that this happens +only once `SectionsManager` has also initialised; the feed can use the method +`SectionsManager.onceInitialized()`. + +### Disabling the section + +The section feed must have an `uninit` method. This is called when the section +feed is disabled by turning the section's pref off. In `uninit` the feed must +call `SectionsManager.disableSection(id)`. This will remove the section's UI +component from every existing Activity Stream page. + +### Updating the section rows + +The section feed can call `SectionsManager.updateSection(id, options)` to update +section options. The `rows` array property of `options` stores the cards of +sites to display. Each card object may have the following properties: + +```js +{ + type, // One of the types in Card/types.js, e.g. "Trending" + title, // Title string + description, // Description string + image, // Image url + url // Site url +} +``` diff --git a/browser/components/newtab/docs/v2-system-addon/telemetry.md b/browser/components/newtab/docs/v2-system-addon/telemetry.md new file mode 100644 index 0000000000..848c931717 --- /dev/null +++ b/browser/components/newtab/docs/v2-system-addon/telemetry.md @@ -0,0 +1,10 @@ +# Telemetry checklist + +Adding telemetry generally involves a few steps: + +1. File a "user story" bug about who wants what question answered. This will be used to track the client-side implementation as well as the data review request. If the server side changes are needed, ask Nan (:nanj / @ncloudio) if in doubt, bugs will be filed separately as dependencies. +1. Implement as usual... +1. Get review from Nan on the data schema and the documentation changes. +1. Request `data-review` of your documentation changes from a [data steward](https://wiki.mozilla.org/Firefox/Data_Collection) to ensure suitability for collection controlled by the opt-out `datareporting.healthreport.uploadEnabled` pref. Download and fill out the [data review request form](https://github.com/mozilla/data-review/blob/master/request.md) and then attach it as a text file on Bugzilla so you can r? a data steward. We've been working with Chris H-C (:chutten) for the Firefox specific telemetry, and Kenny Long (kenny@getpocket.com) for the Pocket specific telemetry, they are the best candidates for the review work as they know well about the context. +1. After landing the implementation, check with Nan to make sure the pings are making it to the database. +1. Once data flows in, you can build dashboard for the new telemetry on [Redash](https://sql.telemetry.mozilla.org/dashboards). If you're looking for some help about Redash or dashboard building, Nan is the guy for that. diff --git a/browser/components/newtab/docs/v2-system-addon/tippytop.md b/browser/components/newtab/docs/v2-system-addon/tippytop.md new file mode 100644 index 0000000000..37111135c7 --- /dev/null +++ b/browser/components/newtab/docs/v2-system-addon/tippytop.md @@ -0,0 +1,40 @@ +# TippyTop in Activity Stream +TippyTop, a collection of icons from the Alexa top sites, provides high quality images for the Top Sites in Activity Stream. The TippyTop manifest is hosted on S3, and then moved to [Remote Settings](https://remote-settings.readthedocs.io/en/latest/index.html) since Firefox 63. In this document, we'll cover how we produce and manage TippyTop manifest for Activity Stream. + +## TippyTop manifest production +TippyTop manifest is produced by [tippy-top-sites](https://github.com/mozilla/tippy-top-sites). + +```sh +# set up the environment, only needed for the first time +$ pip install -r requirements.txt +$ python make_manifest.py --count 2000 > icons.json # Alexa top 2000 sites +``` + +Because the manifest is hosted remotely, we use another repo [tippytop-service](https://github.com/mozilla-services/tippytop-service) for the version control and deployment. Ask :nanj or :r1cky for permission to access this private repo. + +## TippyTop manifest publishing +For each new manifest release, firstly you should tag it in the tippytop-service repo, then publish it as follows: + +### For Firefox 62 and below +File a deploy bug with the tagged version at Bugzilla as [Activity Streams: Application Servers](https://bugzilla.mozilla.org/enter_bug.cgi?product=Firefox&component=Activity%20Streams%3A%20Application%20Servers), assign it to our system engineer :jbuck, he will take care of the rest. + +### For Firefox 63 and beyond +Activity Stream started using Remote Settings to manage TippyTop manifest since Firefox 63. To be able to publish new manifest, you need to be in the author&reviewer group of Remote Settings. See more details in this [mana page](https://mana.mozilla.org/wiki/pages/viewpage.action?pageId=66655528). You can also ask :nanj or :leplatram to get this set up for you. +To publish the manifest to Remote Settings, go to the tippytop-service repo, and run the script as follows, + +```sh +# set up the remote setting, only needed for the first time +$ python3 -m venv .venv +$ source .venv/bin/activate +$ pip install -r requirements.txt + +# publish it to prod +$ source .venv/bin/activate +# It will ask you for your LDAP user name and password. +$ ./upload2remotesettings.py prod +``` + +After uploading it to Remote Setting, you can request for review in the [dashboard](https://remote-settings.mozilla.org/v1/admin/). Note that you will need to log in the Mozilla LDAP VPN for both uploading and accessing Remote Setting's dashboard. Once your request gets approved by the reviewer, the new manifest will be content signed and published to production. + +## TippyTop Viewer +You can use this [viewer](https://mozilla.github.io/tippy-top-sites/manifest-viewer/) to load all the icons in the current manifest. diff --git a/browser/components/newtab/docs/v2-system-addon/unit_testing_guide.md b/browser/components/newtab/docs/v2-system-addon/unit_testing_guide.md new file mode 100644 index 0000000000..c3dd369a2d --- /dev/null +++ b/browser/components/newtab/docs/v2-system-addon/unit_testing_guide.md @@ -0,0 +1,149 @@ +# Unit testing + +## Overview + +Our unit tests in Activity Stream are written with mocha, chai, and sinon, and run +with karma. They include unit tests for both content code (React components, etc.) +and `.jsm`s. + +You can find unit tests in `tests/unit`. + +## Execution + +To run the unit tests once, execute `npm test`. + +To run unit tests continuously (i.e. in "test-driven development" mode), you can +run `npm run tddmc`. + +## Debugging + +To debug tests, you should run them in continuous mode with `npm run tddmc`. +In the Firefox window that is opened (it should say "Karma... - connected"), +click the "debug" button and open your console to see test output, set +breakpoints, etc. + +Unfortunately, source maps for tests do not currently work in Firefox. If you need +to see line numbers, you can run the tests with Chrome by running +`npm install --save-dev karma-chrome-launcher && npm run tddmc -- --browsers Chrome` + +## Where to put new tests + +If you are creating a new test, add it to a subdirectory of the `tests/unit` +that corresponds to the file you are testing. Tests should end with `.test.js` or +`.test.jsx` if the test includes any jsx. + +For example, if the file you are testing is `lib/Foo.jsm`, the test +file should be `test/unit/lib/Foo.test.js` + +## Mocha tests + +All our unit tests are written with [mocha](https://mochajs.org), which injects +globals like `describe`, `it`, `beforeEach`, and others. It can be used to write +synchronous or asynchronous tests: + +```js +describe("FooModule", () => { + // A synchronous test + it("should create an instance", () => { + assert.instanceOf(new FooModule(), FooModule); + }); + describe("#meaningOfLife", () => { + // An asynchronous test + it("should eventually get the meaning of life", async () => { + const foo = new FooModule(); + const result = await foo.meaningOfLife(); + assert.equal(result, 42); + }); + }); +}); +``` + +## Assertions + +To write assertions, use the globally available `assert` object (this is provided +by karma-chai, so you do not need to `require` it). + +For example: + +```js +assert.equal(foo, 3); +assert.propertyVal(someObj, "foo", 3); +assert.calledOnce(someStub); +``` + +You can use any of the assertions from: + +- [`chai`](http://chaijs.com/api/assert/). +- [`sinon-chai`](https://github.com/domenic/sinon-chai#assertions) + +### Custom assertions + +We have some custom assertions for checking various types of actions: + +#### `.isUserEventAction(action)` + +Asserts that a given `action` is a valid User Event, i.e. that it contains only +expected/valid properties for User Events in Activity Stream. + +```js +// This will pass +assert.isUserEventAction(ac.UserEvent({event: "CLICK"})); + +// This will fail +assert.isUserEventAction({type: "FOO"}); + +// This will fail because BLOOP is not a valid event type +assert.isUserEventAction(ac.UserEvent({event: "BLOOP"})); +``` + +## Overriding globals in `.jsm`s + +Most `.jsm`s you will be testing use `Cu.import` or `XPCOMUtils` to inject globals. +In order to add mocks/stubs/fakes for these globals, you should use the `GlobalOverrider` +utility in `test/unit/utils`: + +```js +const {GlobalOverrider} = require("test/unit/utils"); +describe("MyModule", () => { + let globals; + let sandbox; + beforeEach(() => { + globals = new GlobalOverrider(); + sandbox = globals.sandbox; // this is a sinon sandbox + // This will inject a "AboutNewTab" global before each test + globals.set("AboutNewTab", {override: sandbox.stub()}); + }); + // globals.restore() clears any globals you added as well as the sinon sandbox + afterEach(() => globals.restore()); +}); +``` + +## Testing React components + +You should use the [enzyme](https://github.com/airbnb/enzyme) suite of test utilities +to test React Components for Activity Stream. + +Where possible, use the [shallow rendering method](https://github.com/airbnb/enzyme/blob/master/docs/api/shallow.md) (this will avoid unnecessarily +rendering child components): + +```js +const React = require("react"); +const {shallow} = require("enzyme"); + +describe("<Foo>", () => { + it("should be hidden by default", () => { + const wrapper = shallow(<Foo />); + assert.isTrue(wrapper.find(".wrapper").props().hidden); + }); +}); +``` + +If you need to, you can also do [Full DOM rendering](https://github.com/airbnb/enzyme/blob/master/docs/api/mount.md) +with enzyme's `mount` utility. + +```js +const React = require("react"); +const {mount} = require("enzyme"); +... +const wrapper = mount(<Foo />); +``` |