diff options
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/Changelog | 550 | ||||
| -rw-r--r-- | doc/DESIGN-AND-ROADMAP.md | 276 | ||||
| -rw-r--r-- | doc/Translation.md | 92 | ||||
| -rw-r--r-- | doc/admin-deployment.md | 49 | ||||
| -rw-r--r-- | doc/develop.md | 45 | ||||
| -rw-r--r-- | doc/fixing-broken-sites.md | 63 | ||||
| -rw-r--r-- | doc/jid1-MnnxcxisBPnSXQ@jetpack.json | 11 | ||||
| -rw-r--r-- | doc/permissions.md | 27 | ||||
| -rw-r--r-- | doc/sample-managed-storage-manifest-chrome.json | 12 | ||||
| -rw-r--r-- | doc/tests.md | 92 | ||||
| -rw-r--r-- | doc/yellowlist-criteria.md | 8 |
11 files changed, 1225 insertions, 0 deletions
diff --git a/doc/Changelog b/doc/Changelog new file mode 100644 index 0000000..6e36b14 --- /dev/null +++ b/doc/Changelog @@ -0,0 +1,550 @@ +Privacy Badger Release Notes +============================ +2020.10.7 +* Disabled learning (by default) to address privacy concerns. +Visit https://www.eff.org/badger-evolution to learn more. +* Added support for Global Privacy Control, a new specification that +lets you tell companies you'd like to opt out of data sharing and selling. +Visit https://globalprivacycontrol.org/ to learn more. +* Added a new section to the options page to display the browser settings +that Privacy Badger overrides for privacy reasons +* Fixed various site breakages +* Improved translations (Simplified Chinese, Dutch, Finnish, Hebrew, Italian, +Russian, Spanish, Ukrainian) + +2020.8.25 +* Added a button to widget replacements to always allow a widget on a site +* Improved scrolling of tracking domains on the options page +* Fixed various site breakages +* Improved translations (Simplified Chinese, Traditional Chinese, Esperanto, +Finnish, French, Polish, Brazilian Portuguese, Russian, Spanish, Swedish) + +2020.7.21 +* Improved broken site reporting screen in the popup +* Refreshed the look of options page tabs +* Fixed various site breakages +* Improved translations (Simplified Chinese, Danish, European Portuguese, +Spanish, Turkish) + +2020.6.29 +* Added replacement placeholder for Disqus comments widgets +* Fixed domain sliders on the options page not saving in some cases +* Fixed slider changes on the options page resetting the list of domains, +causing you to lose your place if you were scrolled down +* Fixed domain slider tooltip display +* Re-enabled custom tooltips in Firefox +* Fixed various site breakages +* Improved translations (Esperanto, Hebrew, Ukrainian) + +2020.6.2 +* Added replacement placeholders for Facebook Comments/Video and Twitch Player +* Removed Twitter link unwrapping. We are unable to unwrap t.co links +on Twitter at this time, as the original URL is no longer present +in the Twitter website's document structure. +* Fixed various site breakages +* Improved translations (Hebrew, Polish, Swedish) + +2020.5.12 +* Made buttons in the popup easier to see and click (or tap, on Firefox for +Android) +* Added a replacement placeholder widget for Google reCAPTCHA +* Fixed various site breakages +* Improved translations (Simplified Chinese, Dutch, Finnish, French, German, +Hebrew, Italian, Brazilian Portuguese, Spanish, Swedish, Turkish) + +2020.2.19 +* Added website breakage warnings, shown in the popup when you block a domain +known to break websites +* Removed pixel cookie sharing detection pending security fixes +* Fixed various site breakages +* Improved translations (Catalan, Simplified Chinese, Danish, Dutch, German, +Russian) + +2020.1.13 +* Fixed bug that sometimes loses pre-trained data for new users +* Fixed various site breakages + +2020.1.7.1 +* Added helpful text to popup on disabled sites +* Fixed display issues in popup on smaller displays +* Fixed Facebook link unwrapping on messenger.com +* Fixed some cookies getting incorrectly flagged as high entropy +* Fixed various site breakages +* Improved translations (Simplified Chinese, Traditional Chinese, French, +Hebrew, Italian, Korean, Russian, Spanish, Swedish, Ukrainian) + +2019.11.18 +* Improved display of domains Privacy Badger hasn't yet learned to block +* Fixed export/import of the WebRTC protection setting +* Fixed certain YouTube ("video unavailable"), Vimeo ("Because of its privacy +settings, this video cannot be played here"), and other video players +by revising referrer protection for "cookieblocked" domains +* Fixed various other site breakages +* Added Korean translations +* Improved translations (Finnish, French, Hebrew, Swedish) + +2019.10.28 +* Refreshed the popup with a higher-contrast look +* Widget placeholders will no longer be applied for domains on the yellowlist. +The value of widget placeholders is full blocking (best privacy) combined with +a clear way to restore potentially useful blocked content (convenience). +Mixing cookie blocking and placeholders doesn't actually improve privacy or +convenience but does introduce various bugs. +* Added a replacement placeholder for YouTube (disabled by default as youtube.com +is still on the yellowlist for now) +* Added the Widget Replacement tab to the options page to manage widgets that +do get blocked and replaced with Privacy Badger placeholders. Visit Widget +Replacement to selectively disable placeholders. For example, you want social +buttons out of your life completely. +* Fixed various site breakages +* Added Hebrew translations +* Improved translations (Italian, Russian, Ukrainian) + +2019.10.8 +* Fixed image/video thumbnails in Google search results +* Fixed various other site breakages +* Removed the green "0" tracker count badge. +No need to draw attention when there is nothing to show. +* Improved translations (Simplified Chinese, Ukrainian) + +2019.9.23 +* Added helpful text to popup on special browser pages like the New Tab page +* Fixed pixel cookie sharing detection being broken by First-Party Isolation +in Firefox +* Fixed major issues with Service Workers-powered sites like Gmail and Twitter +* Fixed various other site breakages +* Improved translations (Simplified Chinese, French, Spanish, Swedish, +Turkish, Ukrainian) + +2019.7.1 +* Added pixel cookie sharing detection. Privacy Badger now records +tracking by images with querystrings that contain first-party cookie data. +This catches Google Analytics. +* Removed display of non-tracking domains from the popup by default. +Hiding domains that Privacy Badger does not consider to be tracking +should reduce self-inflicted Web breakage. +* Enabled Facebook link unwrapping on messenger.com +* Fixed various site breakages +* Added Catalan translations +* Improved translations (Bulgarian, Traditional Chinese, Persian, +Brazilian Portuguese, European Portuguese, Swedish, Ukrainian) + +2019.2.19 +* Improved replacement widgets: + + - Replaced the "play" icon with an "allow once" button to improve + accessibility and to make it more clear our widgets are interactive + - Made activation also activate any other widgets of the same type + - Updated the replacement for Vimeo to ignore background videos + - Set minimum dimensions to avoid becoming too small or hidden + +* Fixed various site breakages + +2019.1.30 +* Added replacement widgets for embedded Spotify, Streamable and Vimeo +players. Privacy Badger can replace potentially useful third-party widgets +with placeholders. This avoids on-by-default tracking while providing a clear +way to restore the original widget on demand. +* Fixed various site breakages +* Added Arabic and European Portuguese translations +* Improved translations (Traditional Chinese, Czech, German, Persian, Swedish, +Turkish) + +2018.12.17 +* Fixed major Privacy Badger breakages in Chrome 72+ +* Fixed various site breakages +* Improved translations (Simplified Chinese, Spanish, Turkish) + +2018.12.5 +* Added a Share button to the popup. This lets you easily copy and paste +Badger's findings on any page. +* Updated link protection to work on all Google Search country domains +* Updated link protection on Facebook to remove the new "fbclid" tracking +parameter +* Added support for enterprise/admin/group policy settings overrides. This +lets administrators preconfigure Privacy Badger installations. For more +information, please visit +https://github.com/EFForg/privacybadger/blob/master/doc/admin-deployment.md +* Fixed various bugs with local storage protection for "cookieblocked" (slider +set to "yellow") domains +* Made the options page work better on small and on large displays +* Fixed various site breakages +* Improved translations (Simplified Chinese, Dutch, Esperanto, Finnish, +German, Italian, Persian, Polish, Spanish, Swedish) + +2018.10.3.1 +* Fixed style problems with Google Search results +* Fixed the setting to open results in new browser windows on Google Search + +2018.10.3 +* Added protection against outgoing link click tracking on Google Search, +Google Docs and Google Hangouts +* Fixed various site breakages +* Added Finnish translations +* Improved translations (Simplified Chinese) + +2018.9.20 +* Added buttons to back up and restore the disabled sites list using +Firefox/Google Sync. The new buttons live under the Manage Data tab +on the options page. +* Added saving of in-progress error reports so that you no longer lose +your typing when you close the popup for whatever reason +* Fixed popup layout problems when opened in the overflow menu in Firefox +* Updated popup and options to use a higher resolution Badger logo on higher +pixel density displays +* Fixed problems with broken fonts and images on Google Docs +* Improved translations (Traditional Chinese, French, German, +Brazilian Portuguese, Russian, Spanish, Swedish, Turkish) + +2018.8.22 +* Added pre-trained tracker data for new Privacy Badger installations. +Visit www.eff.org/badger-pretraining to learn more. +* Added reset/clear tracker data buttons to the Manage Data options page tab +* Fixed various site breakages +* Added Persian and Brazilian Portuguese translations +* Improved translations (Simplified Chinese, Danish, Esperanto, French, +German, Italian, Norwegian Bokmål, Spanish, Swedish) + +2018.8.1 +* Fixed security issues with HTML5 local storage tracking detection as well as +SoundCloud widget replacement. Thanks again to Cure53 for discovering and +reporting these vulnerabilities. +* Improved Facebook link unwrapping; now enabled on the Facebook Onion domain +* Improved translations (Italian, Norwegian Bokmål, Swedish, Ukrainian) + +2018.7.18.1 +* Added setting to disable sending Do Not Track to websites +* Fixed security issue with link unwrapping on Facebook. Thanks to Cure53 for +discovering and reporting this vulnerability. +* Improved ordering of domain names in the popup and on the options page +* Improved handling of disabled sites with wildcards +* Added t.co link replacement to user profiles on Twitter +* Linked to EFF software privacy policy from the new user welcome page +* Updated to latest dummy Google Tag Manager script from uBlock Origin +to avoid "failed to redirect a network request" warnings in Chrome +* Fixed various site breakages +* Improved translations (Simplified Chinese, Dutch, Esperanto, German, +Russian, Swedish, Turkish) + +2018.5.10 +* Added protection against outgoing link click tracking on Facebook +* Updated WebRTC protection to revert to browser default (off) when disabled +* Fixed popup in the Italian locale in Chrome +* Updated to latest dummy Google Analytics script from uBlock Origin +to avoid "failed to redirect a network request" warnings in Chrome +* Fixed various site breakages +* Improved translations (Traditional Chinese, Danish, Dutch, Esperanto, +French, German, Italian, Polish, Russian, Swedish, Ukrainian) + +2018.4.23 +* Fixed changes not being persisted for domains that appear after scrolling +the tracking domains list on the options page +* Improved tracking domains search on the options page +* Fixed "can't access dead object" errors in Firefox +* Fixed XML document rendering in Firefox +* Updated WebRTC protection checkbox to become disabled when the setting is +controlled by other extensions +* Fixed various site breakages +* Improved translations (Traditional Chinese, Czech, Danish, Esperanto, +French, German, Italian, Polish, Russian, Swedish, Turkish, Ukrainian) + +2018.4.10 +* Updated the new user welcome page. The redesigned page is mobile-friendly, +accessible and already translated into several languages. +* Fixed Do Not Track being checked sometimes in Private Browsing/Incognito +windows. By default, Privacy Badger should not record anything in Incognito. +* Added setting to allow learning in Private Browsing/Incognito windows +* Fixed unwanted scrolling when switching tabs on the options page +* Updated replacement icon for Google+ +* Fixed various site breakages +* Improved translations (Bulgarian, Simplified Chinese, Traditional Chinese, +Danish, Esperanto, French, German, Italian, Polish, Russian, Spanish, +Ukrainian) + +2018.3.21 +* Fixed Do Not Track not being recognized by sites that test for it by +checking navigator.doNotTrack with JavaScript +* Stopped signaling DNT on sites where Privacy Badger is disabled +* Fixed popup in Private Browsing windows in Firefox +* Fixed certain kinds of site breakages (such as visual issues with charts) +not going away even after disabling Privacy Badger on the site +* Updated to latest dummy Google Tag Manager script from uBlock Origin to +avoid "failed to redirect a network request" warnings in Chrome +* Fixed various site breakages +* Improved translations (Bulgarian, Traditional Chinese, German, Italian, +Swedish) + +2018.2.5 +* Added type/status filters to the tracking domains list on the options page +* Reworked social widget replacement to avoid WebExtensions fingerprintability +issue in Firefox +* New translations (Turkish) +* Improved translations (Danish, Esperanto, French, German, Polish, Swedish, +Ukrainian) + +2018.1.30 +* Removed the "unlimitedStorage" permission from the manifest +* Changed the tracker count badge color from red to "Privacy Badger orange" + +2018.1.25 +* Added workaround to avoid Privacy Badger getting disabled as "Not from +Chrome Web Store" in Chrome + +2018.1.22 +* Reduced amount of data stored as part of normal operation. Privacy Badger +will no longer record (or check Do Not Track policies for) non-tracking +domains. This should enable us to remove the "unlimitedStorage" permission +with the next Badger update. +* Improved tracker detection status summaries in popup and options +* Added explanatory acknowledgement to Tracking Domains options page tab +* Fixed file download dialog not showing when exporting user data in Firefox +* Removed tutorial reminder link from popup when already on tutorial page +* Removed "Requests to the server have been blocked by an extension" messages +in Chrome and Opera +* Fixed style issue with replacement social widgets +* Fixed error reporting on pages where Privacy Badger has been disabled +* Fixed various site breakages +* Improved translations (Bulgarian, Danish, Esperanto, French, German, +Italian, Polish, Russian, Spanish, Swedish, Ukrainian) + +2017.11.20 +* Disabled custom tooltips in Firefox to work around browser freezing issues +* Added validation to the disabled sites form +* Improved translations (Dutch, Esperanto and German) +* Added Bulgarian and Polish translations + +2017.11.9 +* Fixed various site breakages +* Improved translations (French, Serbian and Ukrainian) +* Added Esperanto translation + +2017.10.25.1 +* Reverted manifest file change preventing upload to Chrome Web Store + +2017.10.25 +* Added Beta support for Firefox for Android +* Updated popup to close after doing anything that reloads the page +* Improved handling of long domain names +* Improved tooltips +* Restored canvas fingerprinting detection to Firefox +* Fixed yellowlist updates not getting applied when importing Badger data +* Updated to latest dummy Google Analytics script from uBlock Origin +to avoid "failed to redirect a network request" warnings in Chrome +* Fixed various site breakages +* Improved translations (Danish, French, German, Italian, Swedish) + +2017.9.12.1 +* Fixed build script issue that reintroduced major site breakages on Firefox + +2017.9.12 +* Fixed DNT policy checking for blocked domains +* Fixed exporting large Badger datasets +* Made progress on Firefox for Android compatibility +* Fixed various site breakages +* Improved badge updating performance +* Improved translations (Czech, Italian, Swedish, Ukrainian) +* Added Danish translation + +2017.7.24 +* Added validation to yellowlist (f.k.a. "cookieblock list") updating +* Removed faulty yellowlist domain removal logic, which, together with missing +validation and eff.org serving a maintenance page instead of the actual +yellowlist, resulted in major breakages all across the Web, something this +update should prevent from happening ever again +* Fixed various site breakages +* Improved translations (Swedish) + +2017.6.13.1 +* Added workaround for validation issue preventing upload to Chrome Web Store +* Fixed chrome.privacy-related exceptions in Firefox 54 + +2017.6.13 +* Added automatic replacement of t.co shortened tracking URLs with original +unobfuscated URLs on twitter.com +* Added option to disable Do Not Track policy checking +* Restricted DNT policy checking from sending cookies +* Fixed tooltips for DNT-compliant domains in popup +* Fixed localStorage tracking sometimes being attributed to unrelated domains +* Improved translations (Swedish, Ukrainian) + +2017.5.9 +* Improved popup rendering +* Added version number to popup +* Restricted Do Not Track policy checking from being able to set cookies +* Fixed several cookie parsing issues +* Added workaround for Cloudflare security cookies +* Improved translations (Simplified Chinese, Swedish, Ukrainian) + +2017.4.19.1 +* Rework DNT policy rechecking to only happen during browsing. Eliminates +needless rechecking of unlikely-to-be-visited-again domains. Should further +mitigate CPU issues. +* Fix DNT policies to only apply to specific domains they are posted on +* New translations (Ukrainian) +* Improved translations (Simplified Chinese, Italian) +* Fix "trackers" link on popup and options pages +* Fix broken site (sharepoint.com) + +2017.3.28 +* New Translations (Czech) +* Translation Updates +* Fix bug in DNT policy re-checking code +* Rate limit DNT checking to one request per second +* Fix issue with multiple DNT checks at once for a single domain +* Fix cookieblock updating issue +* Fix popup width issue +* Fix DNT hash updating issue +* Fix toggle switch issue +* Automated tests now also run on Firefox +* Other minor bugfixes and broken site fixes + +2017.3.22 +* AMO (Firefox) only release. +* Fix cookie tracking detection in Firefox. + +2017.1.26.1 +* AMO only release +* Fixes an error in the build scripts which reintroduced a firefox bug for AMO users + +2017.1.26 +* Huge speed improvements for settings import and on startup +* Fixes no content blocking bug (firefox) +* Several fixes for broken websites +* Translations fixes +* New Translation: Nordic +* New Translation: Traditional Chineese (Taiwan) +* New Translation: Serbian +* Bugfix: Crash on browsers without WebRTC +* Bugfix: narrow poup if icon is in the menu (firefox) +* Bugfix: Import/Export now uses utf-8 and can handle non english character +sets +* Enhancement: Convert icons to SVG +* Enhancement: Script surrogate for google analytics, gigya, and more... +* KNOWN ISSUE: Chrome will now display a message "Not downloaded from chrome + +2016.12.15.1 (2.0.2) +* BUGFIX: Chrome browsers no longer display privacy badger as (corrupted) +* BUGFIX: Fixes lockup issue on some versions of firefox +* BUGFIX: Fixes issue where privacy badger panel gets cut off +* BUGFIX: Fixes a non implmeneted API in firefox which was causing numerous +sites to break. +* KNOWN ISSUE: Chrome will now display a message "Not downloaded from chrome +store". This is a known side effect of a workaround for a different bug. + +2016.12.8.1 (2.0.1) +* BUGFIX: Sanitize origin and action in popup + +2016.12.8 (2.0) +* BUGFIX: Fix ublock origin warnings +* BUGFIX: Remove need for download permission + +2016.12.7.2 (2.0RC1) +* Huge speed improvements +* Multiprocess Compatible (E10S) for firefox +* Breaks many fewer websites +* Many small bugfixes +* Import and Export your data +* Block WebRTC from leaking your IP address +* Forget data in incognito mode +* block html5 "ping" tracking +* Translation fixes +* (Developers) Firefox and Chrome versions now share one code base! + +2016.9.7 (1.13) +* Add exceptions for multi domain first parties +* Fix google drive download issue +* Fix wikipedia login issue +* Fix youtube comments and notifications issues +* Several other broken site fixes +* Hopefully a fix for the "corrupted extension" issue + +2016.8.29 (1.12) +* UI Tweaks +* Remove last adblock plus code +* Feature: remove domains from list +* Refactor incognito mode handling +* Compatibile with firefox web extensions + +2016.5.24 (1.11) +* Fix build error + +2016.5.23 (1.10) +* Fix cookie block list adding bug +* New migration to fix bug retroactively + +2016.5.16 (1.9) +* Remove Adblock Plus Engine +* Switch to using storage.js and chrome storage API +* Massive refactoring of code +* Huge speed improvements +* Fixes bug where privacy badger "forgets" settings +* Fixes first run tab opening on every startup +* Fix waiting for privacy badger bug +* Fix high CPU usage bug +* Uses separate data store for incognito mode +* Ads selenium test to run pbtest.org sweet +* Fixes weird subdomain handling edge case +* Fixes bug where pages stop loading sometimes + +2015.4.6 (1.8) +* Fix "waiting for privacy badger bug" +* Huge speed improvement + +2015.4.6 (1.7) +* Fix crash when closing options page +* Add EFF Donate Button +* New popup to nag user to go through tutorial + +2015.3.2 (1.0.6) +* New feature: Search within blocked domain list +* Replace soundcloud widget with a click to play button +* Misc. bug fixes and translation improvements + +2015.12.3 (1.0.4) +* Lots of site bug fixes +* Chinese Translation +* Spanish Translation +* italian translation +* UI Overhaul +* Update Swedish locale +* Typo fixes +* Numerous bug fixes +* Added support for disabled sites with wildcards +* Red badge now reflects the number of domains blocked or cookieblocked +instead of all third parties. +* Tooltips show full domain name + +2015.8.14 (1.0.1) +* Fixes a bug where slider settings for a base domain wouldn't take effect +* Fixes 'this extension is slowing down chrome' errors + +2015.8.5 (1.0) +* 1.0 release +* Bugfixes from 2015.7.24 (0.99) +* Detects Canvas Fingerprinting +* Detect Local Storage Supercookies +* Improved UI +* Options page for overriding privacy badger settings +* Report Broken Site button +* Many Bugfixes (see github) +* Translations into swedish, french and german + +2015.7.24 (0.99) +* Release candidate for version 1.0! + +2015.4.1 +* Miscellanious bugfixes +* Improvements to heuristic + +2014.9.16 +* Adds lots of tests including selenium tests. +* Adds lots of domains to the cookie block list. +* Fixes bug with downloading cookie block list. +* Fixes other minor stylistic bugs. + +2014.7.17 +* Created dialog to allow users to unblock certain third parties on certain +* sites for addedd functionality. E.g. disqus comments, facebook comments, etc. +* Added lots of domains to cookie block list. +* do not show domains that do not appear to be trackers in the popup +* added missing google+ button override diff --git a/doc/DESIGN-AND-ROADMAP.md b/doc/DESIGN-AND-ROADMAP.md new file mode 100644 index 0000000..ca96989 --- /dev/null +++ b/doc/DESIGN-AND-ROADMAP.md @@ -0,0 +1,276 @@ +# PRIVACY BADGER DESIGN AND ROADMAP + +## DESIGN + +### OBJECTIVE + +Privacy Badger aims to + + - Protect users against non-consensual tracking by third party domains as they + browse the Web. + + - Send and enforce the Do Not Track signal to sites (especially "third party" + sites since they are in a position to collect a large fraction of the user's + browsing history). + +Privacy Badger consists of a primary tracker blocking algorithm, augmented by +a number of secondary features that extend further privacy protection and +reduce breakage from the primary mechanism. + +### PRIMARY MECHANISM + +Privacy Badger: + +1. Ensures your browser is sending the DNT: 1 header (in some regulatory + environments, it is advisable to note "installing Privacy Badger will enable + Do Not Track" on your installation page / app store entry. +2. Observes which first party origins a given third party origin is setting cookies on + (certain cookies are deemed to be "low entropy", as discussed below). + + 2a. Observes which first party origins a given third party is doing certain + types of fingerprinting on. + + 2b. Observes which first party origins a given third party is setting certain types + of supercookies on. + + 2c. Observes which first party origins a given third party is sending + certain parts of first party cookies back to itself using image query + strings (pixel cookie sharing). + +3. If a third party origin receives a cookie, a supercookie, an image pixel + containing first party cookie data, or makes JavaScript fingerprinting API + calls on 3 or more first party origins, this is deemed to be "cross site + tracking". +4. Typically, cross site trackers are blocked completely; Privacy Badger + prevents the browser from communicating with them. The exception is if the + site is on Privacy Badger's "yellow list" (aka the "cookie block list"), in + which case resources from the site are loaded, but without access to their + (third party) cookies or local storage, and with the referer header either + trimmed down to the origin (for GET requests) or removed outright (all other + requests). The yellow list is routinely fetched from [an EFF URL](https://www.eff.org/files/cookieblocklist_new.txt) + to allow prompt fixes for breakage. + + Until methods for blocking them have been implemented, domains that perform + fingerprinting or use third party supercookies should not be added to the + yellow list. +5. Users can also choose custom rules for any given domain flagged by Privacy Badger, + overrulling any automatic decision Privacy Badger has made about the domain. + Privacy Badger uses three-state sliders (red → block, yellow → cookie block, green → allow) to convey this + state in UI. We believe this is less confusing than the UI in many other + blocking tools, which often leave the user confused about whether a visual + state represents current blocking or the opportunity to block. +6. Domains can agree to EFF's [Do Not Track policy](https://eff.org/dnt-policy). If a domain does this + Privacy Badger will no longer block its traffic or cookies. If a + first-party domain posts the policy, this applies to all third parties + embedded on that domain. + Sites post the policy at [a well-known URL](https://example.com/.well-known/dnt-policy.txt) + on their domains. The contents must match those of a file from the list of + acceptable policies exactly; the policy file is [maintained on github](https://github.com/EFForg/dnt-policy/), + but Privacy Badger fetches a list of known-good hashes periodically [from EFF](https://www.eff.org/files/dnt-policies.json) + (version 1.0 of the policy file will be added to that list when Privacy Badger + reaches version 1.0) + +#### Further Details + +# :warning: THIS SECTION IS OUTDATED AND NEEDS TO BE REWRITTEN :warning: + +Data Structures: + +- action_map = { 'google.com': blocked, 'fonts.google.com': 'cookieblocked', 'apis.fonts.google.com': 'user_cookieblock', 'foo.tracker.net': 'allow', 'tracker.net': 'DNT', } +- snitch_map = {google.com: array('cooperq.com', 'noah.com', 'eff.org'), tracker.net: array(a.com, b.com, c.com)} +- dnt_domains = array('tracker.net', 'dnt.eff.org') +- settings = {social_widgets = true, ...} +- cookie_block_list = "{'fonts.google.com': true, 'maps.google.com', true}" + + +On Request(): + + if privacy badger is not enabled for the tab domain then return + if fqdn is not a third party then return + + action = check_action(fqdn) (described below) + + if action is block then cancel request + if action is cookie_block then strip headers + if fqdn is nontracking (i.e check_action returned nothing) then do nothing + if action is noaction or any user override then async_check_tracking + if action is allow && count == 2 then blocking_check_tracking + if check_tracking changed action then call check_action again + else do_nothing + + async_check_dnt(fqdn) + +check_action(fqdn): returns action + + related_domains = array() + best_action = 'noaction' + + for $domain in range(fqdn ... etld+1) + if action_map contains $domain + related_domains.shift($domain) + + for each domain in related domains + if score(domain.action) > score(best_action) + best_action = domain.action + + return best_action + +check_tracking(fqdn): return boolean + + var base_domain = etld+1(fqdn) + + if has_cookie or has_supercookie or has_fingerprinting + if snitch_map doesn't have base domain add it + if snitch_map doesn't have first party add it + if snitch_map.base_domain.len >= 3 + add base domain to action map as blocked + add all chlidren of base_domain and self from yellow list to action map + return true + +##### What is an "origin" for Privacy Badger? + +Privacy Badger has two notions of origin. One is the [effective top level +domain](https://wiki.mozilla.org/Public_Suffix_List) plus one level of +subdomain (eTLD+1), computed using +[getBaseDomain](https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XPCOM/Reference/Interface/nsIEffectiveTLDService) +(which is built-in to Firefox; in Chrome we [ship a +copy](https://github.com/EFForg/privacybadger/blob/8e8ad9838b74b6d13354163f78d362ca60dd44f9/src/lib/basedomain.js#L75). +The accounting for which origins are trackers or not is performed by looking +up how many first party fully qualified domain names (FQDNs) have been tracked by each +of these eTLD + 1 origins. This is a conservative choice, which avoids the +need to evaluate sets of cookies with different scopes. + +When the heuristic determines that the correct response is to block, +that decision is applied to the third party eTLD+1 from which tracking was seen. + +Users are able to override Privacy Badger's decision for any given FQDN if they +do not wish to block something that is otherwise blocked (or block something +that is not blocked). + +To illustrate this, suppose the site <tt>tracking.co.uk</tt> was embedded on +every site on the Web, but each embed came from a randomly selected subdomain +<tt>a.tracking.co.uk</tt>, <tt>b.tracking.co.uk</tt>, +<tt>c.tracking.co.uk</tt>, etc. Suppose the user visits +<tt>www.news-example.com</tt> and <tt>search.jobs-example.info</tt>. + +The accounting data structure <tt>seenThirdParties</tt> would come to include: + +``` +{ + ... + "tracking.co.uk" : { + "news-example.com" : true, + "jobs-example.info" : true, + } + ... +} +``` + +Now suppose the user visits a third site, <tt>clickbait.nonprofit.org</tt>, +and is tracked by <tt>q.tracking.co.uk</tt> on that site. The +seenThirdParties data structure will have a third entry added to it, meeting +the threshold of three first party origins and defining +<tt>tracking.co.uk</tt> as a tracking eTLD+1. At this point +<tt>tracking.co.uk</tt> will be added to the block list. Any future requests to +<tt>tracking.co.uk</tt>, or any of its subdomains, will be blocked. +The user can manually unblock specific subdomains as necessary via the popup menu. + +##### What is a "low entropy" cookie? + +Our [current cookie heuristic](https://github.com/EFForg/privacybadger/blob/8e8ad9838b74b6d13354163f78d362ca60dd44f9/src/js/heuristicblocking.js#L632) is to assign "number of identifying bits" estimates to +some known common cookie values, and to bound the sum of these to 12. +Predetermined low-entropy cookies will not be identified as tracking, nor will +combinations of them so long as their total estimated entropy is under 12 bits. + +### ADDITIONAL MECHANISMS + +#### Widget Substitution + +Many social media widgets are inherently designed to combine tracking +and occasionally-useful functionality in a single resource load. +Privacy Badger aims to give the user access to the functionality when they want +it, but protection against the tracking at all other times. + +To that end, Privacy Badger has incorporated code from the ShareMeNot project +so that it is able to replace various types of widgets hosted +by third party origins with local, static equivalents that either replace the +original widget faithfully, or create a click-through step before the widget +is loaded and tracks the user. + +The widget replacement table lives in the [socialwidgets.json file](https://github.com/EFForg/privacybadger/blob/8e8ad9838b74b6d13354163f78d362ca60dd44f9/src/data/socialwidgets.json). +Widgets are replaced unless the user has chosen to specifically allow that third party +domain (by moving the slider to 'green' in the UI), so users can selectively +disable this functionality if they wish. The code for social media widgets is +quite diverse, so not all variants (especially custom variants that sites build +for themselves) are necessarily replaced. + +#### What are the states for domain responses? + +Currently domains have three states: no action, cookie block, and block. No +action allows all requests to resolve as normal without intervention from +Privacy Badger. Cookie block allows for requests to resolve normally but will +block cookies from being read or created. Cookie block also trims or removes +the referer header. Block will cause any requests from that origin to be +blocked entirely; before even a TCP connection can be established. The user can +toggle these options manually, which will supersede any determinations made +automatically by Privacy Badger. + +#### What does EFFs Do Not Track policy stipulate? + +Currently the Do Not Track policy covers where the agreement will be hosted, +how users who send the DNT header are treated, log retention, how information +will be shared with other domains, notifications of disclosure, and possible exceptions. +It can be read in full [here](https://www.eff.org/dnt-policy). + +#### How do sites agree to EFFs Do Not Track policy? + +Sites can agree to this policy by posting at https://subdomain.example.com/.well-known/dnt-policy.txt, +where "subdomain" is any domain to which the policy applies, for a given third party. + +#### Fingerprinting detection +Certain aspects of the browser, such as fonts, add-ons or extensions, screen size, +and seen links, can be used to give the browser a fingerprint that is unique out +of a very small amount of users (see [Panopticlick](https://panopticlick.eff.org/) for more information). + +As of Privacy Badger 1.0, any third party script that writes to an HTML5 +canvas object and then reads a sufficiently large amount back from the third +party canvas object will be treated the same way as a third party cookie, blocking the +third party origin if it does this across multiple first party origins. Our +research has determined that this is a reliable way to distinguish between +fingerprinting and other third party canvas uses. + +This may be augmented by hooks to detect extensive enumeration of properties +in the <tt>navigator</tt> object in the future. + +#### Pixel cookie sharing detection + +Detection of first to third party cookie sharing via image pixels was added in [#2088](https://github.com/EFForg/privacybadger/issues/2088). + +### ROADMAP + +#### High priority issues + +Please see our ["high priority"-labeled issues](https://github.com/EFForg/privacybadger/issues?q=is%3Aissue+is%3Aopen+label%3A%22high+priority%22). + +## Technical Implementation + +### How are origins and the rules for them stored? + +When a browser with Privacy Badger enabled makes a request to a third party, if +the request contains a cookie or the response tries to set a cookie it gets +flagged as 'tracking'. Origins that make tracking requests get stored in a +key→value store where the keys are the origins making the request, and the +values are the first party origins these requests were made on. If that list of +third parties contains three or more first party origins the third party origin +gets added to another list of known trackers. When Privacy Badger gets a +request from an origin on the known trackers list, if it is not on the yellow +list then Privacy Badger blocks that request. If it is on the yellow list then +the request is allowed to resolve, but all cookie setting and getting parts of +it are blocked, while the referer header is trimmed or removed. Both of these +lists are stored on disk, and persist between browser sessions. + +Additionally users can manually set the desired action for any FQDN. +These get added to their own lists, which are also stored on disk, and get checked +before Privacy Badger does its default action for a given origin. These are managed +from the popup window for Privacy Badger on the page as well as the options menu +for the whole extension. diff --git a/doc/Translation.md b/doc/Translation.md new file mode 100644 index 0000000..6292b07 --- /dev/null +++ b/doc/Translation.md @@ -0,0 +1,92 @@ +# Translating Privacy Badger + +We need your help in translating Privacy Badger to every possible language! + +When translating you should always use the source (American English) locale as +the reference. You can also use existing translations from other languages to +help you in case of doubt, but you should always consider the English version +as the correct one. + + +#### A note about adding translation strings in PRs + +While working on a Privacy Badger enhancement, you might need to add one or +more localized strings. You only need to add new strings to the source +(`en_US`) locale. There is no need to manually add untranslated copies of new +messages to all other locales. This will be taken care of later by a Privacy +Badger maintainer. + + +## Working with translations on GitHub + +Translations on GitHub are done with JSON files. +Each language has its own folder inside +[`src/_locales/`](https://github.com/EFForg/privacybadger/tree/master/src/_locales) +(e.g. 'es' for Spanish, 'ru' for Russian, etc.). +Inside each of these folders is a JSON file that contains the translated +strings for that language. Each entry in the JSON file follows this structure: + + "string_identifier": { + "message": "String text" + "description": "Some useful info" + } + +The translated string is the `"String text"` part. You should **NOT** change +any other part of the entry. + +The `"Some useful info"` part sometimes contains useful information (in +English) about the string. Usually it provides the context of the string: what +it is ("a section heading") and where it can be found in the UI ("on the new +user intro page"). You should not translate it. + +To contribute on GitHub, first check the status of your local language +translation: if you don't see a folder with your +[local language code](https://developer.chrome.com/webstore/i18n?csw=1#localeTable), +the translation for that language is missing. In this case you should follow +the instructions below to set up the JSON file for your language. If the +translation for your language is already there, you can contribute by checking +its accuracy and by correcting any errors you find (see below for +instructions). + +#### Add a new language + +To add a new language on GitHub, follow these steps: + +1. Fork this repository +2. Inside your fork, create a folder in `src/_locales/` and name it +with your [local language code](https://developer.chrome.com/webstore/i18n?csw=1#localeTable) +3. Copy the `src/_locales/en_US/messages.json` file to the folder you created +4. Start translating each message to your language by replacing the +English strings with the translated ones +5. When you have completed the translation, [open a pull request](https://help.github.com/articles/creating-a-pull-request-from-a-fork/). Here you can find +an example translation pull request: [#1270](https://github.com/EFForg/privacybadger/pull/1270). + +#### Correct an existing translation + +To correct errors in an existing translation: + +1. Fork this repository +2. Open your local language JSON file and apply the changes +3. When you have completed your work, [open a pull request](https://help.github.com/articles/creating-a-pull-request-from-a-fork/). +Here you can find an example translation pull request: +[#1270](https://github.com/EFForg/privacybadger/pull/1270). + + +## Testing your translations + +To see your (in-progress) translations in the actual Privacy Badger UI, you should first [load Privacy Badger from source code](/doc/develop.md#install-from-source). + +A quick/hacky way to change Privacy Badger's locale is to temporarily copy the locale you want to use to your default (OS) locale's folder in `src/_locales/` and reload Privacy Badger. + +The proper way would be to launch the browser in your desired locale. + +For Chrome, it might be as easy as [launching it from the command line with `LANGUAGE=fr` (for example) in front of the executable](https://stackoverflow.com/questions/24992240/start-google-chrome-with-a-specific-locale-using-a-command-line-argument). + +Firefox requires [downloading a language pack](https://addons.mozilla.org/en-US/firefox/language-tools/) and [setting it as your locale from about:config](https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/Internationalization#Testing_out_your_extension). + + +## Other information + +To learn about outstanding translations-related issues, and to +see how translations have been handled in the past, take a look +at issues and pull requests marked with the [translations label](https://github.com/EFForg/privacybadger/issues?utf8=%E2%9C%93&q=label%3Atranslations%20). diff --git a/doc/admin-deployment.md b/doc/admin-deployment.md new file mode 100644 index 0000000..41c1d0e --- /dev/null +++ b/doc/admin-deployment.md @@ -0,0 +1,49 @@ +# Privacy Badger enterprise deployment and configuration + +Administrators can configure Privacy Badger on managed devices by setting up a policy. + +You can find the full list of available settings in [Privacy Badger's managed storage schema](/src/data/schema.json). Please [let us know](https://privacybadger.org/#I-found-a-bug%21-What-do-I-do-now) if you'd like to set something that isn't yet supported. + +Note that Privacy Badger currently reads and applies settings from [managed storage](https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/storage/managed) on startup. To see your policy take effect on a managed device, first restart that device's browser. + + +## Firefox setup + +1. Locate and if necessary create the [managed storage manifests folder](https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/Native_manifests#Manifest_location). Note that on Windows you need to create a registry key that points to the manifest's location. +2. Copy the [sample managed storage manifest for Firefox](/doc/jid1-MnnxcxisBPnSXQ@jetpack.json) to this folder. + +If your Privacy Badgers were installed from [Privacy Badger's homepage](https://privacybadger.org) (not from [AMO](https://addons.mozilla.org/en-US/firefox/addon/privacy-badger17/)): + +3. Rename the manifest to `jid1-MnnxcxisBPnSXQ-eff@jetpack.json`. +4. Similarly, update the `"name"` property in the manifest to `"jid1-MnnxcxisBPnSXQ-eff@jetpack"`. + + +## Chrome/Chromium setup + +Please review the [Configuring Apps and Extensions by Policy](http://www.chromium.org/administrators/configuring-policy-for-extensions) document. + +Notes for Chrome OS and Linux follow. + +### Chrome OS + +The following example JSON policy disables Privacy Badger on `example.com`. This means Privacy Badger will be disabled when you visit any `example.com` page. + +```json +{ + "disabledSites": { + "Value": [ + "example.com" + ] + }, + "showIntroPage": { + "Value": false + } +} +``` + +### Linux + +1. Locate and if necessary create the [managed policies folder for Chrome or Chromium](http://www.chromium.org/administrators/configuring-policy-for-extensions). +2. Copy the [sample managed storage manifest for Chrome](/doc/sample-managed-storage-manifest-chrome.json) to this folder. +3. Rename the manifest file to whatever you like (perhaps `privacy-badger-admin-settings.json`). +4. Update the extension ID inside the manifest if you are not using official Privacy Badger releases from Chrome Web Store. diff --git a/doc/develop.md b/doc/develop.md new file mode 100644 index 0000000..cd28b16 --- /dev/null +++ b/doc/develop.md @@ -0,0 +1,45 @@ +# Working with Privacy Badger's code + +To make changes to Privacy Badger, you have to first load the extension from a source code checkout. + + +## Install from source + +To install Privacy Badger from source in Chrome, visit `chrome://extensions`, enable "Developer mode", click "Load unpacked" and select the [`src`](src/) subdirectory inside your copy of the Privacy Badger source code. + +In Firefox, visit `about:debugging`, click "Load Temporary Add-on" and select the [`src/manifest.json`](src/manifest.json) file. Note that this only installs the extension temporarily; it will be removed when you close Firefox. + +To install Privacy Badger from source in Firefox for Android, please see [Mozilla's guide to developing extensions for Firefox for Android](https://extensionworkshop.com/documentation/develop/developing-extensions-for-firefox-for-android/) and [`web-ext` documentation](https://extensionworkshop.com/documentation/develop/getting-started-with-web-ext/#testing-in-firefox-for-android). + + +## Send a pull request + +Before submitting a pull request (PR), please review the sections below. + +### Style guide + +All JavaScript code going forward should use the following naming conventions: + +- Objects and their properties should be Java or camelCase style. +- Primitive types should be Python or snake_case style. +- Constants should be ALL_CAPS. + +Examples: + +```javascript +const TRACKER_ENTROPY_THRESHOLD = 33; + +let tab_id = details.tabId; + +window.badger.getTrackerCount(tab_id); +``` + +### Catch errors early with static code analysis + +First, install the exact expected version of [ESLint](https://eslint.org) by running `npm install` in your Privacy Badger source code checkout directory. You should then be able to produce a lint report by running `make lint` in the same directory. + +You can review our set of ESLint rules in [`.eslintrc.yml`](/.eslintrc.yml). Files we want ESLint to ignore are specified in [`.eslintignore`](/.eslintignore). + +### Commit messages + +Please review the suggestions in this excellent [guide to writing commit messages](https://chris.beams.io/posts/git-commit/). diff --git a/doc/fixing-broken-sites.md b/doc/fixing-broken-sites.md new file mode 100644 index 0000000..7a31da9 --- /dev/null +++ b/doc/fixing-broken-sites.md @@ -0,0 +1,63 @@ +# How to fix broken site issues + +Unfortunately, while working to protect your privacy, Privacy Badger can end up breaking website functionality. Here are the [open "broken site" and "help wanted"-labeled issues](https://github.com/EFForg/privacybadger/issues?utf8=✓&q=is%3Aissue%20is%3Aopen%20label%3A"broken%20site"%20label%3A"help%20wanted"%20). + +This document is about the process of classifying and resolving these breakages. + + +## Confirm Privacy Badger is responsible + +The first thing to do is to confirm that Privacy Badger blocks (or will eventually learn to block) the domain, and that blocking the domain does indeed break the site. + +Browser caching can get in our way here, as cached resources bypass request filtering by extensions. Disable your browser cache when debugging, for example by reloading using <kbd>Ctrl+Shift+R</kbd> every time. + +Try disabling Privacy Badger for the site, and then reloading the page. Does that fix the issue? If it doesn't, does disabling the entire Privacy Badger add-on and reloading the page fix the issue? If it still doesn't, Privacy Badger is not at fault. + +If disabling Badger and reloading the page fixed the issue, and re-enabling and reloading brought the issue back, let's try to figure out the responsible domain(s). Try allowing half the blocked domains to load. If (after reloading the page) the issue was fixed, pick half of those domains and revert them back to Badger's control. Eventually you should find the exact domain(s) that, when blocked, cause the issue to appear. + + +## Resolve the breakage + +Once the issue is confirmed (and the responsible domains have been identified), you should try to find the most appropriate way to resolve it. Privacy Badger comes with several approaches: + +| Approach | Label | Original issue | Difficulty | Notes | +| --- | :---: | :---: | :---: | --- | +| Multi-domain first parties | [MDFP](https://github.com/EFForg/privacybadger/labels/MDFP) | [#781](https://github.com/EFForg/privacybadger/issues/781) | Easy | Narrowly applicable | +| Script surrogates | [surrogates](https://github.com/EFForg/privacybadger/labels/surrogates) | [#400](https://github.com/EFForg/privacybadger/issues/400) | Hard | Should use uBlock Origin's surrogates ("neutered scripts") as much as possible | +| Widget replacement | [widgets](https://github.com/EFForg/privacybadger/labels/widgets) | [#196](https://github.com/EFForg/privacybadger/issues/196), [#1467](https://github.com/EFForg/privacybadger/issues/1467) | Medium | Still needs review/improvements, although some progress being made ([#2262](https://github.com/EFForg/privacybadger/pull/2262)) | +| EFF's Do Not Track policy | [DNT Policy](https://github.com/EFForg/privacybadger/labels/DNT%20policy)| - | n/a | Narrowly applicable | +| Yellowlisting | [yellowlist](https://github.com/EFForg/privacybadger/labels/yellowlist)| - | Easy | Only protects against some types of tracking | + +The question to ask is, which way addresses the issue most specifically, resolving the breakage while increasing privacy exposure by the smallest amount? If you are not sure, that's OK! Opening a new issue (or chiming in on an existing issue) to ask for help is fine. + +Let's look at some common kinds of breakages and see how they relate to the approaches above. + + +### Domains that are part of the site but don't look like it + +Does the blocked domain actually belong to the site, but Privacy Badger doesn't know that and so treats the domain as an external tracker? Sounds like a job for [multi-domain first parties](https://github.com/EFForg/privacybadger/issues/781) (MDFP). + +When adding domains to the MDFP list, please add base ([eTLD](https://en.wikipedia.org/wiki/Public_Suffix_List)+1) domains only. For example, there is no need to add `api.example.net` when adding `example.com` and `example.net`. + +For past examples, you could browse [the list of merged pull requests with the "MDFP" label](https://github.com/EFForg/privacybadger/issues?q=label%3AMDFP+is%3Amerged). + + +### JavaScript errors + +Does blocking the domain block a JavaScript analytics library that the site tries to use and fails, breaking site navigation? This could be resolved by [script surrogates](https://github.com/EFForg/privacybadger/issues/400). + + +### External services + +Is the missing comments section powered by a commenting service that Privacy Badger learned to block? Perhaps a new [widget replacement](https://github.com/EFForg/privacybadger/pull/196) should be added. + +We should also ask the service to to adopt the [EFF Do Not Track policy](https://www.eff.org/dnt-policy), which is a way for privacy-conscious companies to receive recognition for their good practices. If their service can and will abide by the policy's requirements, posting the policy on the service's domains will tell Privacy Badger to allow loading of resources from those domains. + + +### External domains too complex to surrogate or replace with placeholders + +If nothing else seems to fit, adding the affected domain to the "[yellowlist](/doc/yellowlist-criteria.md)" will make Privacy Badger set the domain to "yellow" ("cookie-blocked") instead of "red" (blocked) after seeing it track on three or more sites. + +Resources from yellowlisted domains are requested without referrer headers, and are restricted from reading or writing cookies or localStorage. + +[Here is an example yellowlist pull request](https://github.com/EFForg/privacybadger/pull/1543) that shows what's good to know when deciding how to fix a breakage, and how to get that information. diff --git a/doc/jid1-MnnxcxisBPnSXQ@jetpack.json b/doc/jid1-MnnxcxisBPnSXQ@jetpack.json new file mode 100644 index 0000000..b9242ca --- /dev/null +++ b/doc/jid1-MnnxcxisBPnSXQ@jetpack.json @@ -0,0 +1,11 @@ +{ + "name": "jid1-MnnxcxisBPnSXQ@jetpack", + "description": "This is a sample Firefox managed storage manifest.", + "type": "storage", + "data": { + "showIntroPage": false, + "disabledSites": [ + "example.com" + ] + } +} diff --git a/doc/permissions.md b/doc/permissions.md new file mode 100644 index 0000000..1211b2a --- /dev/null +++ b/doc/permissions.md @@ -0,0 +1,27 @@ +# Permissions + +This document explains the need for each [extension permission](https://developer.chrome.com/extensions/declare_permissions) declared in Privacy Badger's [extension manifest](/src/manifest.json). + +## Privacy +The Privacy API lets extensions modify browser-wide privacy settings. Privacy Badger uses it to disable a setting that lets Chrome send third-party requests to resolve errors, and to turns off link tracking via the HTML ping attribute. We also give users the option to change their WebRTC privacy level in order to prevent leaking local network address information. + +## Cookies +Privacy Badger needs access to the cookies API in order to detect and correct a common error where Cloudflare domains are identified as trackers and blocked. + +## Storage +The storage API lets extensions store information that persists after the browser is closed. Privacy Badger uses it to save user settings and information it has learned about trackers. + +## WebRequest +The WebRequest API allows extensions to observe all incoming and outgoing network requests made by the browser. Privacy Badger inspects request for tracking behavior, and logs the destinations of outgoing requests that are flagged as tracking. No information is ever shared outside of the browser. + +## WebRequestBlocking +The blocking version of the WebRequest API allows extensions to modify or block network requests before they leave the browser. Privacy Badger uses this API to synchronously view, modify, and block requests to trackers. For example, Privacy Badger modifies requests made to domains on the yellowlist to remove the referer header and cookies. + +## WebNavigation +This API allows extensions to detect when the user navigates from one web page to another. Privacy Badger needs this in order to correctly determine whether each request is a first-party request (to the same domain as the web page) or a third-party request (to somewhere else). This permission allows it to avoid misattributing trackers on special pages such as Service Worker pages. + +## http://\*/\* and https://\*/\* +These permissions allow Privacy Badger to use the WebRequest and WebRequestBlocking permissions on requests to all websites. As described above, Privacy Badger uses these APIs to analyze requests and detect tracking, then modify or block requests to known trackers. No information is ever shared outside of the browser. + +## Tabs +Privacy Badger needs access to the tabs API so that the extension can detect which tab is active and which tabs are simply present in the background. The extension icon, badge and popup update to reflect the state of Privacy Badger. This often requires knowing the tab's URL. For example, updating the icon requires the URL in order to determine whether Privacy Badger should be shown as disabled on that tab. Privacy Badger also uses the tabs API for miscellaneous tasks such as opening or switching to the already open new user welcome page. diff --git a/doc/sample-managed-storage-manifest-chrome.json b/doc/sample-managed-storage-manifest-chrome.json new file mode 100644 index 0000000..194bd02 --- /dev/null +++ b/doc/sample-managed-storage-manifest-chrome.json @@ -0,0 +1,12 @@ +{ + "3rdparty": { + "extensions": { + "pkehgijcmpdhfbdbbnkijodmdjhbjlgp": { + "showIntroPage": false, + "disabledSites": [ + "example.com" + ] + } + } + } +} diff --git a/doc/tests.md b/doc/tests.md new file mode 100644 index 0000000..b03c4a3 --- /dev/null +++ b/doc/tests.md @@ -0,0 +1,92 @@ +# Working with Privacy Badger's tests + +We have a few different types of tests: + +* We use [unit tests](/doc/tests.md#unit-tests) for confirming that smaller pieces of code behave as expected. +* [Functional tests](/doc/tests.md#functional-tests) test the UI and that things integrate together properly. +* [Travis CI](/doc/tests.md#travis-ci) runs all these automatically for every pull request on both Chrome and Firefox. + +## Travis CI + +Every pull request runs the full suite of functional and unit tests on [Travis CI](https://travis-ci.org/). We test on latest stable Chrome and Firefox releases, as well as on Chrome Beta, Firefox Beta and Firefox ESR. + +See [`.travis.yml`](/.travis.yml) for Travis configuration, [`scripts/setup_travis.sh`](/scripts/setup_travis.sh) for test setup, and [`scripts/run_travis.sh`](/scripts/run_travis.sh) for test execution procedures. + +We use [ESLint](https://eslint.org) to flag potential JavaScript errors and style issues. Please see our [developer guide](/doc/develop.md#lint-your-changes) for setup instructions. + +## Unit tests + +We use [QUnit](https://qunitjs.com/) for unit tests. +Unit tests are defined in [`/src/tests/tests`](/src/tests/tests). Unit test dependencies live in [`/src/tests/lib`](/src/tests/lib). + +To run unit tests, first [load Privacy Badger from source code](/doc/develop.md#install-from-source) (as we don't ship unit tests with published versions). +Once you loaded Badger from source, click on its button in your browser toolbar to open Badger's popup. +Then in the popup, click on the gear icon (⚙) to open the options page. +Your browser should navigate to an internal URL that starts with `chrome-extension://` or `moz-extension://` and ends with `/skin/options.html`. +Replace `/skin/options.html` with `/tests/index.html` and hit <kbd>Enter</kbd>. +This will open the unit test suite and run the tests. + +### Writing unit tests + +When writing unit tests, try to scope each test to the function or method in question, then each individual assertion within that test addressing a core piece of functionality or expectation of that test. Consider testing expected input, potential breaking points, and expected outputs. It's easy to get caught going down rabbit holes testing unlikely scenarios, so consider which edge cases are most important to consider, and which are more likely to occur. + +Do verify that removing or mutating the code being tested produces failed assertions. + +## Functional tests + +Our [functional tests](/tests/selenium/) are written in Python and driven by [Selenium](https://selenium-python.readthedocs.io/) and [pytest](https://docs.pytest.org/en/latest/). + +To run them in Chrome, you need to [install `chromedriver`](http://chromedriver.chromium.org/getting-started). In Firefox, you need to [install `geckodriver`](https://github.com/EFForg/privacybadger/blob/1550b9efb64c1d5e276361e3940f402c3ec87afc/scripts/setup_travis.sh#L21-L50). + +You also need to [install the Python packages](https://snarky.ca/a-quick-and-dirty-guide-on-how-to-install-packages-for-python/) specified in [`/tests/requirements.txt`](/tests/requirements.txt). + +You should now be able to run the Selenium tests. Try them out by running +the code below. This should take several minutes. +```bash +$ BROWSER=chrome pytest -v tests/ +``` + +macOS users may need to provide the full path to the browser application folder. For example, to run tests on macOS: +```bash +$ BROWSER=/Applications/Firefox.app/Contents/MacOS/firefox-bin pytest -v tests/ +# or +$ BROWSER=/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome pytest -v tests/ +``` + +For more information, see our Travis CI [setup](/scripts/setup_travis.sh) and +[run](/scripts/run_travis.sh) scripts. + + +### Invocation examples + +Note that to use a debugger like `pdb` or `ipdb` you must pass the `-s` (`--capture=no`) flag to pytest. +```bash +# run qunit_test.py, with Firefox, with verbose output (-v) +$ BROWSER=/usr/bin/firefox pytest -v tests/selenium/qunit_test.py tests/ + +# run a specific test on a specific class in a specific module with Chrome Beta +$ BROWSER=google-chrome-beta pytest tests/selenium/super_cookie_test.py::SupercookieTest::test_should_detect_ls_of_third_party_frame + +# run any tests whose name (including the module and class) matches the string cookie_test +# this is often useful as a less verbose way to run a single test +$ BROWSER=firefox pytest -k cookie_test tests/ +``` + +More pytest invocations can be found [here](https://docs.pytest.org/en/latest/usage.html). + +If you are on Linux, you can also run the tests headlessly (without displaying a GUI). +Install `Xvfb` with your system package manager, then set the `ENABLE_XVFB=1` environment variable: + +```bash +$ BROWSER=firefox ENABLE_XVFB=1 pytest -s -v -k PopupTest tests/ +``` + +### Writing functional tests + +Test methods that you want to be discovered and run by `pytest` must be prefixed with the keyword `test`. For example: `test_pixel_tracking_detection`. A similar rule applies to naming any new test class files that you want to be detected by the testing suite: the `test` keyword must be appended to the end of the title. For example: `pixel_test.py`. + +When testing Badger's tracker detection/learning, you should first clear the pre-trained/seed tracker data. For example (run on Badger's options page): `self.js("chrome.extension.getBackgroundPage().badger.storage.clearTrackerData();")`. Clearing seed data ensures that the tracking domain was discovered just now and not from seed data. + +You should also set up your tracking detection test in a way where your test fixture has a "no tracking" mode that you visit first and assert that no tracking was detected. This is to ensure that when we detect the tracking being tested we didn't actually detect some other kind of tracking instead. + +Just as with unit tests, please verify that removing or mutating the code being tested produces failed assertions. diff --git a/doc/yellowlist-criteria.md b/doc/yellowlist-criteria.md new file mode 100644 index 0000000..c209328 --- /dev/null +++ b/doc/yellowlist-criteria.md @@ -0,0 +1,8 @@ +EFF maintains a Privacy Badger "yellowlist" of domains for which requests are allowed but Privacy Badger restricts access or availability of objectionable cookies and potentially other objectionable identifiers. + +Our objective in curating that list is to maximize user privacy while minimizing disruption to functionality that users expect from sites. The criteria we examine when considering possible yellowlist entries include (but are not limited to): + +* Was this in [Bau and Mayer's](https://jonathanmayer.org/papers_data/bau13.pdf) manually curated non-tracker list? +* Is this domain necessary for functionality the user expects from 1st party pages? +* Is the domain's privacy policy clear that it does not perform non-consensual tracking? +* Is there a reasonable self-hosted surrogate available that could replace the functionality of this domain (e.g. https://github.com/EFForg/privacybadgerchrome/issues/400). |