Adding upstream version 2020.10.7.upstream/2020.10.7upstream

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

1 files changed, 276 insertions, 0 deletions

diff --git a/doc/DESIGN-AND-ROADMAP.md b/doc/DESIGN-AND-ROADMAP.md
new file mode 100644
index 0000000..ca96989
--- /dev/null
+++ b/doc/DESIGN-AND-ROADMAP.md
@@ -0,0 +1,276 @@
+# PRIVACY BADGER DESIGN AND ROADMAP
+
+## DESIGN
+
+### OBJECTIVE
+
+Privacy Badger aims to
+
+ - Protect users against non-consensual tracking by third party domains as they
+   browse the Web.
+
+ - Send and enforce the Do Not Track signal to sites (especially "third party"
+   sites since they are in a position to collect a large fraction of the user's
+   browsing history).
+
+Privacy Badger consists of a primary tracker blocking algorithm, augmented by
+a number of secondary features that extend further privacy protection and
+reduce breakage from the primary mechanism.
+
+### PRIMARY MECHANISM
+
+Privacy Badger:
+
+1. Ensures your browser is sending the DNT: 1 header (in some regulatory
+   environments, it is advisable to note "installing Privacy Badger will enable
+   Do Not Track" on your installation page / app store entry.
+2. Observes which first party origins a given third party origin is setting cookies on
+   (certain cookies are deemed to be "low entropy", as discussed below).
+
+   2a. Observes which first party origins a given third party is doing certain
+   types of fingerprinting on.
+
+   2b. Observes which first party origins a given third party is setting certain types
+   of supercookies on.
+
+   2c. Observes which first party origins a given third party is sending
+   certain parts of first party cookies back to itself using image query
+   strings (pixel cookie sharing).
+
+3. If a third party origin receives a cookie, a supercookie, an image pixel
+   containing first party cookie data, or makes JavaScript fingerprinting API
+   calls on 3 or more first party origins, this is deemed to be "cross site
+   tracking".
+4. Typically, cross site trackers are blocked completely; Privacy Badger
+   prevents the browser from communicating with them. The exception is if the
+   site is on Privacy Badger's "yellow list" (aka the "cookie block list"), in
+   which case resources from the site are loaded, but without access to their
+   (third party) cookies or local storage, and with the referer header either
+   trimmed down to the origin (for GET requests) or removed outright (all other
+   requests). The yellow list is routinely fetched from [an EFF URL](https://www.eff.org/files/cookieblocklist_new.txt)
+   to allow prompt fixes for breakage.
+
+   Until methods for blocking them have been implemented, domains that perform
+   fingerprinting or use third party supercookies should not be added to the
+   yellow list.
+5. Users can also choose custom rules for any given domain flagged by Privacy Badger,
+   overrulling any automatic decision Privacy Badger has made about the domain.
+   Privacy Badger uses three-state sliders (red → block, yellow → cookie block, green → allow) to convey this
+   state in UI. We believe this is less confusing than the UI in many other
+   blocking tools, which often leave the user confused about whether a visual
+   state represents current blocking or the opportunity to block.
+6. Domains can agree to EFF's [Do Not Track policy](https://eff.org/dnt-policy). If a domain does this
+   Privacy Badger will no longer block its traffic or cookies. If a
+   first-party domain posts the policy, this applies to all third parties
+   embedded on that domain.
+   Sites post the policy at [a well-known URL](https://example.com/.well-known/dnt-policy.txt)
+   on their domains. The contents must match those of a file from the list of
+   acceptable policies exactly; the policy file is [maintained on github](https://github.com/EFForg/dnt-policy/),
+   but Privacy Badger fetches a list of known-good hashes periodically [from EFF](https://www.eff.org/files/dnt-policies.json)
+   (version  1.0 of the policy file will be added to that list when Privacy Badger
+   reaches version 1.0)
+
+#### Further Details
+
+# :warning: THIS SECTION IS OUTDATED AND NEEDS TO BE REWRITTEN :warning:
+
+Data Structures:
+
+- action_map = { 'google.com': blocked, 'fonts.google.com': 'cookieblocked', 'apis.fonts.google.com': 'user_cookieblock', 'foo.tracker.net': 'allow', 'tracker.net': 'DNT', }
+- snitch_map = {google.com: array('cooperq.com', 'noah.com', 'eff.org'), tracker.net: array(a.com, b.com, c.com)}
+- dnt_domains = array('tracker.net', 'dnt.eff.org')
+- settings = {social_widgets = true, ...}
+- cookie_block_list = "{'fonts.google.com': true, 'maps.google.com', true}"
+
+
+On Request():
+
+      if privacy badger is not enabled for the tab domain then return
+      if fqdn is not a third party then return
+
+      action = check_action(fqdn) (described below)
+
+      if action is block then cancel request
+      if action is cookie_block then strip headers
+      if fqdn is nontracking (i.e check_action returned nothing) then do nothing
+      if action is noaction or any user override then async_check_tracking
+      if action is allow && count == 2 then blocking_check_tracking
+        if check_tracking changed action then call check_action again
+        else do_nothing
+
+      async_check_dnt(fqdn)
+
+check_action(fqdn): returns action
+
+      related_domains = array()
+      best_action = 'noaction'
+
+      for $domain in range(fqdn ... etld+1)
+        if action_map contains $domain
+          related_domains.shift($domain)
+
+        for each domain in related domains
+          if score(domain.action) > score(best_action)
+            best_action = domain.action
+
+        return best_action
+
+check_tracking(fqdn): return boolean
+
+      var base_domain = etld+1(fqdn)
+
+      if has_cookie or has_supercookie or has_fingerprinting
+        if snitch_map doesn't have base domain add it
+        if snitch_map doesn't have first party add it
+        if snitch_map.base_domain.len >= 3
+          add base domain to action map as blocked
+          add all chlidren of base_domain and self from yellow list to action map
+          return true
+
+##### What is an "origin" for Privacy Badger?
+
+Privacy Badger has two notions of origin.  One is the [effective top level
+domain](https://wiki.mozilla.org/Public_Suffix_List) plus one level of
+subdomain (eTLD+1), computed using
+[getBaseDomain](https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XPCOM/Reference/Interface/nsIEffectiveTLDService)
+(which is built-in to Firefox; in Chrome we [ship a
+copy](https://github.com/EFForg/privacybadger/blob/8e8ad9838b74b6d13354163f78d362ca60dd44f9/src/lib/basedomain.js#L75).
+The accounting for which origins are trackers or not is performed by looking
+up how many first party fully qualified domain names (FQDNs) have been tracked by each
+of these eTLD + 1 origins.  This is a conservative choice, which avoids the
+need to evaluate sets of cookies with different scopes.
+
+When the heuristic determines that the correct response is to block,
+that decision is applied to the third party eTLD+1 from which tracking was seen.
+
+Users are able to override Privacy Badger's decision for any given FQDN if they
+do not wish to block something that is otherwise blocked (or block something
+that is not blocked).
+
+To illustrate this, suppose the site <tt>tracking.co.uk</tt> was embedded on
+every site on the Web, but each embed came from a randomly selected subdomain
+<tt>a.tracking.co.uk</tt>, <tt>b.tracking.co.uk</tt>,
+<tt>c.tracking.co.uk</tt>, etc.  Suppose the user visits
+<tt>www.news-example.com</tt> and <tt>search.jobs-example.info</tt>.
+
+The accounting data structure <tt>seenThirdParties</tt> would come to include:
+
+```
+{
+  ...
+  "tracking.co.uk" : {
+    "news-example.com"  : true,
+    "jobs-example.info" : true,
+  }
+  ...
+}
+```
+
+Now suppose the user visits a third site, <tt>clickbait.nonprofit.org</tt>,
+and is tracked by <tt>q.tracking.co.uk</tt> on that site.  The
+seenThirdParties data structure will have a third entry added to it, meeting
+the threshold of three first party origins and defining
+<tt>tracking.co.uk</tt> as a tracking eTLD+1.  At this point
+<tt>tracking.co.uk</tt> will be added to the block list. Any future requests to
+<tt>tracking.co.uk</tt>, or any of its subdomains, will be blocked.
+The user can manually unblock specific subdomains as necessary via the popup menu.
+
+##### What is a "low entropy" cookie?
+
+Our [current cookie heuristic](https://github.com/EFForg/privacybadger/blob/8e8ad9838b74b6d13354163f78d362ca60dd44f9/src/js/heuristicblocking.js#L632) is to assign "number of identifying bits" estimates to
+some known common cookie values, and to bound the sum of these to 12.
+Predetermined low-entropy cookies will not be identified as tracking, nor will
+combinations of them so long as their total estimated entropy is under 12 bits.
+
+### ADDITIONAL MECHANISMS
+
+#### Widget Substitution
+
+Many social media widgets are inherently designed to combine tracking
+and occasionally-useful functionality in a single resource load.
+Privacy Badger aims to give the user access to the functionality when they want
+it, but protection against the tracking at all other times.
+
+To that end, Privacy Badger has incorporated code from the ShareMeNot project
+so that it is able to replace various types of widgets hosted
+by third party origins with local, static equivalents that either replace the
+original widget faithfully, or create a click-through step before the widget
+is loaded and tracks the user.
+
+The widget replacement table lives in the [socialwidgets.json file](https://github.com/EFForg/privacybadger/blob/8e8ad9838b74b6d13354163f78d362ca60dd44f9/src/data/socialwidgets.json).
+Widgets are replaced unless the user has chosen to specifically allow that third party
+domain (by moving the slider to 'green' in the UI), so users can selectively
+disable this functionality if they wish. The code for social media widgets is
+quite diverse, so not all variants (especially custom variants that sites build
+for themselves) are necessarily replaced.
+
+#### What are the states for domain responses?
+
+Currently domains have three states: no action, cookie block, and block. No
+action allows all requests to resolve as normal without intervention from
+Privacy Badger. Cookie block allows for requests to resolve normally but will
+block cookies from being read or created. Cookie block also trims or removes
+the referer header. Block will cause any requests from that origin to be
+blocked entirely; before even a TCP connection can be established. The user can
+toggle these options manually, which will supersede any determinations made
+automatically by Privacy Badger.
+
+#### What does EFFs Do Not Track policy stipulate?
+
+Currently the Do Not Track policy covers where the agreement will be hosted,
+how users who send the DNT header are treated, log retention, how information
+will be shared with other domains, notifications of disclosure, and possible exceptions.
+It can be read in full [here](https://www.eff.org/dnt-policy).
+
+#### How do sites agree to EFFs Do Not Track policy?
+
+Sites can agree to this policy by posting at https://subdomain.example.com/.well-known/dnt-policy.txt,
+where "subdomain" is any domain to which the policy applies, for a given third party.
+
+#### Fingerprinting detection
+Certain aspects of the browser, such as fonts, add-ons or extensions, screen size,
+and seen links, can be used to give the browser a fingerprint that is unique out
+of a very small amount of users (see [Panopticlick](https://panopticlick.eff.org/) for more information).
+
+As of Privacy Badger 1.0, any third party script that writes to an HTML5
+canvas object and then reads a sufficiently large amount back from the third
+party canvas object will be treated the same way as a third party cookie, blocking the
+third party origin if it does this across multiple first party origins. Our
+research has determined that this is a reliable way to distinguish between
+fingerprinting and other third party canvas uses.
+
+This may be augmented by hooks to detect extensive enumeration of properties
+in the <tt>navigator</tt> object in the future.
+
+#### Pixel cookie sharing detection
+
+Detection of first to third party cookie sharing via image pixels was added in [#2088](https://github.com/EFForg/privacybadger/issues/2088).
+
+### ROADMAP
+
+#### High priority issues
+
+Please see our ["high priority"-labeled issues](https://github.com/EFForg/privacybadger/issues?q=is%3Aissue+is%3Aopen+label%3A%22high+priority%22).
+
+## Technical Implementation
+
+### How are origins and the rules for them stored?
+
+When a browser with Privacy Badger enabled makes a request to a third party, if
+the request contains a cookie or the response tries to set a cookie it gets
+flagged as 'tracking'. Origins that make tracking requests get stored in a
+key→value store where the keys are the origins making the request, and the
+values are the first party origins these requests were made on. If that list of
+third parties contains three or more first party origins the third party origin
+gets added to another list of known trackers. When Privacy Badger gets a
+request from an origin on the known trackers list, if it is not on the yellow
+list then Privacy Badger blocks that request. If it is on the yellow list then
+the request is allowed to resolve, but all cookie setting and getting parts of
+it are blocked, while the referer header is trimmed or removed. Both of these
+lists are stored on disk, and persist between browser sessions.
+
+Additionally users can manually set the desired action for any FQDN.
+These get added to their own lists, which are also stored on disk, and get checked
+before Privacy Badger does its default action for a given origin. These are managed
+from the popup window for Privacy Badger on the page as well as the options menu
+for the whole extension.