1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
|
====================
Introduction to Sync
====================
This document is a brief introduction to how Sync is implemented in desktop Firefox.
General, Historical, Anatomy of a Sync Engine
=============================================
This section describes how Sync used to work - and indeed, how much of it still
does. While we discuss how this is slowly changing, this context is valuable.
For any datatype which syncs, there tends to be 3 parts:
Store
-----
The sync ``store`` interfaces with the actual Firefox desktop store. For example,
in the ``passwords`` engine, the "store" is that layer that talks to
``Services.logins``
Tracker
-------
The ``tracker`` is what knows that something should be synced. For example,
when the user creates or updates a password, it is the tracker that knows
we should sync now, and what particular password(s) should be updated.
This is typically done via "observer" notifications - ``Services.logins``,
``places`` etc all send specific notifications when certain events happen
(and indeed, some of these were added for Sync's benefit)
Engine
------
The ``engine`` ties it all together. It works with the ``store`` and
``tracker`` and tracks its own metadata (eg, the timestamp of the passwords on
the server, so it knows how to grab just changed records and how to pass them
off to the ``store`` so the actual underlying storage can be updated.
All of the above parts were typically in the
`services/sync/modules/engines directory <https://searchfox.org/mozilla-central/source/services/sync/modules/engines>`_
directory and decoupled from the data they were syncing.
The Future of Desktop-Specific Sync Engines
===========================================
The system described above reflects the fact that Sync was "bolted on" to
Desktop Firefox relatively late - eg, the Sync ``store`` is decoupled from the
actual ``store``. This has causes a number of problems - particularly around
the ``tracker`` and the metadata used by the engine, and the fact that changes
to the backing store would often forget that Sync existed.
Over the last few years, the Sync team has come to the conclusion that Sync
support must be integrated much closer to the store itself. For example,
``Services.logins`` should track when something has changed that would cause
an item to be synced. It should also track the metadata for the store so that
if (say) a corrupt database is recovered by creating a new, empty one, the
metadata should also vanish so Sync knows something bad has happened and can
recover.
However, this is a slow process - currently the ``bookmarks``, ``history`` and
``passwords`` legacy engines have been improved so more responsibility is taken
by the stores. In all cases, for implementation reasons, the Sync
implementation still has a ``store``, but it tends to be a thin wrapper around
the actual underlying store.
The Future of Cross-Platform Sync Engines
=========================================
There are a number of Sync engines implemented in Rust and which live in the
application-services repository. While these were often done for mobile
platforms, the longer term hope is that they can be reused on Desktop.
:doc:`engines` has more details on these.
While no existing engines have been replaced with Rust implemented engines,
the webext-storage engine is implemented in Rust via application-services, so
doesn't tend to use any of the infrastructure described above.
Hopefully over time we will find more Rust-implemented engines in Desktop.
|
