1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
|
.. _healthreport_architecture:
============
Architecture
============
``healthreporter.sys.mjs`` contains the main interface for FHR, the
``HealthReporter`` type. An instance of this is created by the
``data_reporting_service``.
``providers.sys.mjs`` contains numerous ``Metrics.Provider`` and
``Metrics.Measurement`` used for collecting application metrics. If you
are looking for the FHR probes, this is where they are.
Storage
=======
Firefox Health Report stores data in 3 locations:
* Metrics measurements and provider state is stored in a SQLite database
(via ``Metrics.Storage``).
* Service state (such as the IDs of documents uploaded) is stored in a
JSON file on disk (via OS.File).
* Lesser state and run-time options are stored in preferences.
Preferences
===========
Preferences controlling behavior of Firefox Health Report live in the
``datareporting.healthreport.*`` branch.
Service and Data Control
------------------------
The follow preferences control behavior of the service and data upload.
service.enabled
Controls whether the entire health report service runs. The overall
service performs data collection, storing, and submission.
This is the primary kill switch for Firefox Health Report
outside of the build system variable. i.e. if you are using an
official Firefox build and wish to disable FHR, this is what you
should set to false to prevent FHR from not only submitting but
also collecting data.
uploadEnabled
Whether uploading of data is enabled. This is the preference the
checkbox in the preferences UI reflects. If this is
disabled, FHR still collects data - it just doesn't upload it.
service.loadDelayMsec
How long (in milliseconds) after initial application start should FHR
wait before initializing.
FHR may initialize sooner than this if the FHR service is requested.
This will happen if e.g. the user goes to ``about:healthreport``.
service.loadDelayFirstRunMsec
How long (in milliseconds) FHR should wait to initialize on first
application run.
FHR waits longer than normal to initialize on first application run
because first-time initialization can use a lot of I/O to initialize
the SQLite database and this I/O should not interfere with the
first-run user experience.
documentServerURI
The URI of a Bagheera server that FHR should interface with for
submitting documents.
You typically do not need to change this.
documentServerNamespace
The namespace on the document server FHR should upload documents to.
You typically do not need to change this.
infoURL
The URL of a page containing more info about FHR, it's privacy
policy, etc.
about.reportUrl
The URL to load in ``about:healthreport``.
about.reportUrlUnified
The URL to load in ``about:healthreport``. This is used instead of ``reportUrl`` for UnifiedTelemetry when it is not opt-in.
service.providerCategories
A comma-delimited list of category manager categories that contain
registered ``Metrics.Provider`` records. Read below for how provider
registration works.
If the entire service is disabled, you lose data collection. This means
that **local** data analysis won't be available because there is no data
to analyze! Keep in mind that Firefox Health Report can be useful even
if it's not submitting data to remote servers!
Logging
-------
The following preferences allow you to control the logging behavior of
Firefox Health Report.
logging.consoleEnabled
Whether to write log messages to the web console. This is true by
default.
logging.consoleLevel
The minimum log level FHR messages must have to be written to the
web console. By default, only FHR warnings or errors will be written
to the web console. During normal/expected operation, no messages of
this type should be produced.
logging.dumpEnabled
Whether to write log messages via ``dump()``. If true, FHR will write
messages to stdout/stderr.
This is typically only enabled when developing FHR.
logging.dumpLevel
The minimum log level messages must have to be written via
``dump()``.
State
-----
currentDaySubmissionFailureCount
How many submission failures the client has encountered while
attempting to upload the most recent document.
lastDataSubmissionFailureTime
The time of the last failed document upload.
lastDataSubmissionRequestedTime
The time of the last document upload attempt.
lastDataSubmissionSuccessfulTime
The time of the last successful document upload.
nextDataSubmissionTime
The time the next data submission is scheduled for. FHR will not
attempt to upload a new document before this time.
pendingDeleteRemoteData
Whether the client currently has a pending request to delete remote
data. If true, the client will attempt to delete all remote data
before an upload is performed.
FHR stores various state in preferences.
Registering Providers
=====================
Firefox Health Report providers are registered via the category manager.
See ``HealthReportComponents.manifest`` for providers defined in this
directory.
Essentially, the category manager receives the name of a JS type and the
URI of a sys.mjs to import that exports this symbol. At run-time, the
providers registered in the category manager are instantiated.
Providers are registered via the category manager to make registration
simple and less prone to errors. Any XPCOM component can create a
category manager entry. Therefore, new data providers can be added
without having to touch core Firefox Health Report code. Additionally,
category manager registration means providers are more likely to be
registered on FHR's terms, when it wants. If providers were registered
in code at application run-time, there would be the risk of other
components prematurely instantiating FHR (causing a performance hit if
performed at an inopportune time) or semi-complicated code around
observers or listeners. Category manager entries are only 1 line per
provider and leave FHR in control: they are simple and safe.
Document Generation and Lifecycle
=================================
FHR will attempt to submit a JSON document containing data every 24 wall
clock hours.
At upload time, FHR will query the database for **all** information from
the last 180 days and assemble this data into a JSON document. We
attempt to upload this JSON document with a client-generated UUID to the
configured server.
Before we attempt upload, the generated UUID is stored in the JSON state
file on local disk. At this point, the client assumes the document with
that UUID has been successfully stored on the server.
If the client is aware of other document UUIDs that presumably exist on
the server, those UUIDs are sent with the upload request so the client
can request those UUIDs be deleted. This helps ensure that each client
only has 1 document/UUID on the server at any one time.
Importance of Persisting UUIDs
------------------------------
The choices of how, where, and when document UUIDs are stored and updated
are very important. One should not attempt to change things unless she
has a very detailed understanding of why things are the way they are.
The client is purposefully very conservative about forgetting about
generated UUIDs. In other words, once a UUID is generated, the client
deliberately holds on to that UUID until it's very confident that UUID
is no longer stored on the server. The reason we do this is because
*orphaned* documents/UUIDs on the server can lead to faulty analysis,
such as over-reporting the number of Firefox installs that stop being
used.
When uploading a new UUID, we update the state and save the state file
to disk *before* an upload attempt because if the upload succeeds but
the response never makes it back to the client, we want the client to
know about the uploaded UUID so it can delete it later to prevent an
orphan.
We maintain a list of UUIDs locally (not simply the last UUID) because
multiple upload attempts could fail the same way as the previous
paragraph describes and we have no way of knowing which (if any)
actually succeeded. The safest approach is to assume every document
produced managed to get uploaded some how.
We store the UUIDs on a file on disk and not anywhere else because we
want storage to be robust. We originally stored UUIDs in preferences,
which only flush to disk periodically. Writes to preferences were
apparently getting lost. We switched to writing directly to files to
eliminate this window.
|
