diff options
Diffstat (limited to 'docs/bug-mgmt')
19 files changed, 1657 insertions, 0 deletions
diff --git a/docs/bug-mgmt/assets/icon_assignments.png b/docs/bug-mgmt/assets/icon_assignments.png Binary files differnew file mode 100644 index 0000000000..81b13703f0 --- /dev/null +++ b/docs/bug-mgmt/assets/icon_assignments.png diff --git a/docs/bug-mgmt/guides/bug-pipeline.rst b/docs/bug-mgmt/guides/bug-pipeline.rst new file mode 100644 index 0000000000..a70adb0464 --- /dev/null +++ b/docs/bug-mgmt/guides/bug-pipeline.rst @@ -0,0 +1,37 @@ +Bug pipeline +============ + +For Firefox quality, Mozilla has different processes to report defects. In parallel, over the years, Mozilla developed many tools around bug management. + +.. mermaid:: + + graph TD + classDef tool fill:#f96; + + Community --> B(bugzilla.mozilla.org) + QA --> B + Foxfooding --> B + Fuzzing --> B + SA[Static/Dynamic analysis] --> B + P[Performance monitoring] --> B + Y[Test automation] --> B + Z[Crash detection] --> B + B --> C{Bug update} + C --> D[Metadata improvements] + C --> E[Component triage] + C --> F[Test case verification] + F --> BM{{Bugmon}}:::tool + F --> MR{{Mozregression}}:::tool + + D --> AN{{Autonag}}:::tool + E --> BB{{bugbug}}:::tool + D --> BB + +More details +------------ + +* :ref:`Fuzzing` +* `Autonag <https://wiki.mozilla.org/Release_Management/autonag#Introduction>`_ - `Source <https://github.com/mozilla/relman-auto-nag/>`_ +* `Bugbug <https://github.com/mozilla/bugbug>`_ - `Blog post about triage <https://hacks.mozilla.org/2019/04/teaching-machines-to-triage-firefox-bugs/>`_ / `Blog post about CI <https://hacks.mozilla.org/2020/07/testing-firefox-more-efficiently-with-machine-learning/>`_ +* `Bugmon <https://hacks.mozilla.org/2021/01/analyzing-bugzilla-testcases-with-bugmon/>`_ - `Source <https://github.com/MozillaSecurity/bugmon>`_ +* `Mozregression <https://mozilla.github.io/mozregression/>`_ - `Source <https://github.com/mozilla/mozregression>`_ diff --git a/docs/bug-mgmt/guides/bug-types.rst b/docs/bug-mgmt/guides/bug-types.rst new file mode 100644 index 0000000000..3626336de1 --- /dev/null +++ b/docs/bug-mgmt/guides/bug-types.rst @@ -0,0 +1,29 @@ +Bug Types +========= + +We organize bugs by type to make it easier to make triage decisions, get +the bug to the right person to make a decision, and understand release +quality. + +- **Defect** regression, crash, hang, security vulnerability and any + other reported issue +- **Enhancement** new feature, improvement in UI, performance, etc. and + any other request for user-facing enhancements to the product, not + engineering changes +- **Task** refactoring, removal, replacement, enabling or disabling of + functionality and any other engineering task + +All bug types need triage decisions. Engineering :ref:`triages defects and +tasks <Triage for Bugzilla>`. Product management :ref:`triages +enhancements <New Feature Triage>`. + +It’s important to distinguish an enhancement from other types because +they use different triage queues. + +Distinguishing between defects and tasks is important because we want to +understand code quality and reduce the number of defects we introduce as +we work on new features and fix existing defects. + +When triaging, a task can be as important as a defect. A behind the +scenes change to how a thread is handled can affect performance as seen +by a user. diff --git a/docs/bug-mgmt/guides/other-metadata.rst b/docs/bug-mgmt/guides/other-metadata.rst new file mode 100644 index 0000000000..f1b94f16d8 --- /dev/null +++ b/docs/bug-mgmt/guides/other-metadata.rst @@ -0,0 +1,28 @@ +Other Bug Metadata +================== + +Performance +----------- + +- Use the ``perf`` keyword +- Add ``[qf:?]`` to the whiteboard if you think the Performance team + should look at this bug + +Privacy +------- + +- Use the ``privacy`` keyword + +User Security +------------- + +- Will this bug adversely affect Firefox users if left public? + + - Add to security group + +- Otherwise move bug to one of: + + - Core:: Security + - Firefox:: Security + - Toolkit:: Security + - Webkit:: Security diff --git a/docs/bug-mgmt/guides/priority.rst b/docs/bug-mgmt/guides/priority.rst new file mode 100644 index 0000000000..db0c8ee874 --- /dev/null +++ b/docs/bug-mgmt/guides/priority.rst @@ -0,0 +1,27 @@ +Priority Definitions +==================== + +We use these definitions across all components: + ++----------------------------------------+-----------------------------+ +| Priority | Description | ++========================================+=============================+ +| \- | No decision | ++----------------------------------------+-----------------------------+ +| P1 | Fix in the current release | +| | cycle | ++----------------------------------------+-----------------------------+ +| P2 | Fix in the next release | +| | cycle or the following | +| | (nightly + 1 or nightly + | +| | 2) | ++----------------------------------------+-----------------------------+ +| P3 | Backlog | ++----------------------------------------+-----------------------------+ +| P4 | Do not use, this priority | +| | is for web platform test | +| | bots | ++----------------------------------------+-----------------------------+ +| P5 | Will not fix, but will | +| | accept a patch | ++----------------------------------------+-----------------------------+ diff --git a/docs/bug-mgmt/guides/severity.rst b/docs/bug-mgmt/guides/severity.rst new file mode 100644 index 0000000000..c75870f471 --- /dev/null +++ b/docs/bug-mgmt/guides/severity.rst @@ -0,0 +1,71 @@ +Defect Severity +=============== + +Definition +---------- + +We use the ``severity`` field in Bugzilla to indicate the scope of a +bug's effect on Firefox. + +The field is display alongside the bug's priority. + +Values +------ + +Severity levels and their definitions are enumerated at https://wiki.mozilla.org/BMO/UserGuide/BugFields#bug_severity. + +By default, new bugs have a severity of ``--``. + +Examples of S1 bugs +^^^^^^^^^^^^^^^^^^^ + +- WebExtensions disabled for all users +- Web search not working from URL bar +- Crashes with data loss + +Examples of S2 bugs +^^^^^^^^^^^^^^^^^^^ + +Bugs that could reasonably be expected to cause a Firefox user to switch browsers, +either because the severity is bad enough, or the frequency of occurrence is high enough. + +- `Bug 1640441 <https://bugzilla.mozilla.org/show_bug.cgi?id=1640441>`__ - Slack hangs + indefinitely in a onResize loop +- `Bug 1645651 <https://bugzilla.mozilla.org/show_bug.cgi?id=1645651>`__ - Changes in + Reddit's comment section JS code makes selecting text slow on Nightly + +Bugs involving contractual partners (if not an S1) + +Bugs reported from QA + +- `Bug 1640913 <https://bugzilla.mozilla.org/show_bug.cgi?id=1640913>`__ - because an + important message is not visible with the dark theme. It's not marked as S1 since the + issue is reproducible only on one OS and the functionality is not affected. +- `Bug 1641521 <https://bugzilla.mozilla.org/show_bug.cgi?id=1641521>`__ - because videos + are not working on several sites with ETP on (default). This is not an S1 since turning + ETP off fixes the issue. + +Fuzzblocker bugs, which prevent fuzzing from making progress + +Examples of S3 bugs +^^^^^^^^^^^^^^^^^^^ + +Bugs filed by contributors as part of daily refactoring and maintenance of the code base. + +`Bug 1634171 <https://bugzilla.mozilla.org/show_bug.cgi?id=1634171>`__ - Visual artifacts around circular images + +Bugs reported from QA + +- `Bug 1635105 <https://bugzilla.mozilla.org/show_bug.cgi?id=1635105>`__ because + the associated steps to reproduce are uncommon, + and the issue is no longer reproducible after refresh. +- `Bug 1636063 <https://bugzilla.mozilla.org/show_bug.cgi?id=1636063>`__ since it's + reproducible only on a specific web app, and only with a particular set of configurations. + + +Rules of thumb +-------------- + +- The severity of most bugs of type ``task`` and ``enhancement`` will be + ``N/A`` +- **Do not** assign bugs of type ``defect`` the severity ``N/A`` diff --git a/docs/bug-mgmt/guides/status-flags.rst b/docs/bug-mgmt/guides/status-flags.rst new file mode 100644 index 0000000000..04d252f596 --- /dev/null +++ b/docs/bug-mgmt/guides/status-flags.rst @@ -0,0 +1,33 @@ +Release Status Flags +==================== + +The flag ``status_firefoxNN`` has many values, here’s a cheat sheet. + +== ========== ========== ============ ================= +— ? unaffected affected fixed +== ========== ========== ============ ================= +? unaffected wontfix verified +\ affected fix-optional disabled +\ fixed verified disabled +== ========== ========== ============ ================= + +The headers of the table are values of the status flag. Each column are +the states reachable from the column headings. + +- ``---`` we don’t know whether Firefox N is affected +- ``?`` we don’t know whether Firefox N is affected, but we want to find + out. +- ``affected`` - present in this release +- ``unaffected`` - not present in this release +- ``fixed`` - a contributor has landed a change set in the tree + to address the issue +- ``verified`` - the fix has been verified by QA or other contributors +- ``disabled`` - the fix or the feature has been backed out or disabled +- ``verified disabled`` - QA or other contributors confirmed the fix or + the feature has been backed out or disabled +- ``wontfix`` - we have decided not to accept/uplift a fix for this + release cycle (it is not the same as the bug resolution WONTFIX). + This can also mean that we don’t know how to fix that and will ship + with this bug +- ``fix-optional`` - we would take a fix for the current release but + don’t consider it as important/blocking for the release diff --git a/docs/bug-mgmt/index.rst b/docs/bug-mgmt/index.rst new file mode 100644 index 0000000000..215b2e7317 --- /dev/null +++ b/docs/bug-mgmt/index.rst @@ -0,0 +1,40 @@ +Bug Handling +============ + +Guides +------ + +.. toctree:: + :maxdepth: 1 + :glob: + + guides/* + +Policies +-------- + +.. toctree:: + :maxdepth: 1 + :glob: + + policies/* + +Processes +--------- + +.. toctree:: + :maxdepth: 1 + :glob: + + processes/* + +Related documentation +--------------------- + +- `bugzilla.mozilla.org documentation <https://bmo.readthedocs.org/>`__ +- `bugzilla.mozilla.org field + definitions <https://wiki.mozilla.org/BMO/UserGuide/BugFields>`__ +- `Lando + documentation <https://moz-conduit.readthedocs.io/en/latest/lando-user.html>`__ +- `Mozilla Phabricator (Code Review) + documentation <https://moz-conduit.readthedocs.io/en/latest/phabricator-user.html>`__ diff --git a/docs/bug-mgmt/policies/attention-dashboard.rst b/docs/bug-mgmt/policies/attention-dashboard.rst new file mode 100644 index 0000000000..b3f79502e9 --- /dev/null +++ b/docs/bug-mgmt/policies/attention-dashboard.rst @@ -0,0 +1,108 @@ +What Needs My Attention +======================= + +Bugzilla’s `What needs my attention?`_ (a.k.a What Should I Work On Next) dashboard helps us to focus on the top-most important or urgent engineering tasks for releasing Firefox. + +The dashboard is not designed for including everything on a person’s plate. It doesn’t attempt to prioritize normal work - it is just for the things that are more important than the normal stuff. It’s about individual engineer prioritization rather than team prioritization and it doesn’t claim to include all sources of high priority work (e.g. triage, responding to requests from HR, etc). + +The dashboard is available to Mozilla engineers, using the icon + + .. image:: ../assets/icon_assignments.png + :alt: Notification Icon + +in the top right hand corner, after you log into Bugzilla. + +The dashboard is a collection of some (sometimes non-obvious) Bugzilla searches. **Web Platform engineers** should check this page once per day, and ideally keep the list empty, so we can focus on our normal or planned work. + + +.. _What needs my attention?: https://bugzilla.mozilla.org/page.cgi?id=whats_next.html + + +Rules of thumb +-------------- + +Here are some rules of thumb about our priorities when it comes to bug fixing. Specific requests from a manager take precedence over these instructions. + + +Code review requests are not visible on this dashboard; please visit `Phabricator`_ to view those. In general it’s reasonable and important to prioritize Review Requests to unblock others. + + +Follow these general principles and use your best judgment with the help of a manager when necessary. + + +.. _Phabricator: https://phabricator.services.mozilla.com/ + + +Highest priority tasks +~~~~~~~~~~~~~~~~~~~~~~ + +These are the things you should drop everything else for. Generally, work where you block others should be addressed as higher priority than non-blocking work. + +#. Critical needinfos + + * Bugs that are needinfo? you and are marked as Severity = S1 defects or with the “sec-critical” keyword. + + * Bugs that are needinfo? you and are marked as being tracked against or blocking the current beta release. These bugs are potentially S2 once they are triaged and important. If we don’t act on these bugs, we’re in danger of delaying a release. + + * Bugs that are needinfo? you and are marked as a security issue without rating. These bugs are potentially sec-critical or sec-high once they are triaged and important. + + +#. Critical bugs assigned to you + + * Severity = S1 defects and bugs with the “sec-critical” keyword. + + +High priority tasks +~~~~~~~~~~~~~~~~~~~ + +High priority tasks are also “drop everything”, except that in this case “everything” doesn’t include anything in the “Highest priority” list. Generally, work where you block others should be addressed as higher priority than non-blocking work. + +#. Important needinfos + + * Bugs that are needinfo? you and are marked as Severity = S2 defects or with the “sec-high” keyword. + + +#. Important bugs assigned to you + + * Severity = S2 defects and bugs with the “sec-high” keyword (both for things that are not disabled in the current release) + * Note: Some teams have very long lists of S2 defects - see notes below on “Long High Priority task lists” + +#. Your other needinfos (except for things that are self-needinfo). + + +Handling needinfos +------------------ + +TL;DR: Don’t leave people hanging. Reply and convert needinfos from others to self-needinfos. + +Many people have long lists of needinfos. Please don’t ignore them. Here’s how we suggest you burn them down, and keep them down: + +* Any needinfo older than 3 months has probably been forgotten about by the requester. It’s okay to declare needinfo bankruptcy. If you’re concerned about annoying the requester by clearing the needinfo, feel free to point them at this document and offer that they can re-request if there is still a problem. + +* For newer requests, don’t leave someone or something, .e.g BugBot hanging. If you can take action in the short term, do so. If you can’t do it straight away, reply letting the requester know that you intend to work on it, but can’t do so immediately. This clears the original needinfo. At the same time you can re-request a needinfo (using the “Request information from ‘myself’” feature). This converts the needinfo to a self-needinfo, and takes it off the attention list. Be mindful, do the conversion when you have given the right attention and response so that others are not hanging on there, or you've done the best you can for the time being to unblock others. We don't want it to become a way to ignore requests. + + +Review requests +--------------- + +This list doesn’t include Review Requests as as we are still investigating the feasibility of including them and applying these strict rules, but we might consider adding this to a future revision. + +In the meanwhile, it’s worthwhile considering the use of peer review groups set up in Phabricator so that multiple engineers can assist in reviews. + + +Other notes +----------- + +* Long lists of High Priority tasks: For some people and teams, the list of “High Priority” tasks is so long that you would never do normal work. If this is you then you should schedule these tasks alongside normal work. However making your task list manageable should still be a priority. + +* `Severity`_ is defined, but things get a bit hazy when it comes to how we define severity for enhancements; this list is for serious defects only. + +.. _Severity: https://firefox-source-docs.mozilla.org/bug-mgmt/guides/severity.html + + +Everything else +--------------- + +This list is not designed for including everything or prioritizing your normal work. Over time we’d like to bring teams’ practices for prioritizing new work more in line with each other, but that’s not the job of this note. + +If you find that most of your time is spent on high or highest priority tasks, then it’s time to ask some questions to work out why - there’s likely to be a problem behind this and it sounds like a recipe for burnout, and we should do everything we can to even things out. diff --git a/docs/bug-mgmt/policies/new-feature-triage.rst b/docs/bug-mgmt/policies/new-feature-triage.rst new file mode 100644 index 0000000000..7bc8022777 --- /dev/null +++ b/docs/bug-mgmt/policies/new-feature-triage.rst @@ -0,0 +1,55 @@ +New Feature Triage +================== + +Identifying Feature Requests +---------------------------- + +Bugs which request new features or enhancements should be of +type=\ ``enhancement``. + +Older bugs may also be feature requests if some or all of the following +are true: + +- Bugs with ``feature`` or similar in whiteboard or short description +- ``[RFE]`` in whiteboard, short description, or description +- Bugs not explicitly marked as a feature request, but appear to be + feature requests +- Bugs marked with ``feature`` keyword + +Initial Triage +-------------- + +Staff, contractors, and other contributors looking at new bugs in +*Firefox::Untriaged* and *::General* components should consider if a +bug, if not marked as a feature request, should be one, and if so: + +- Update the bug’s type to ``enhancement`` +- Determine which product and component the bug belongs to and update + it **or** + + - Use *needinfo* to ask a component’s triage owner or a module’s + owner where the request should go + +Product Manager Triage +---------------------- + +- The product manager for the component reviews bugs of type + ``enhancement`` + + - This review should be done a least weekly + +- Reassigns to another Product::Component if necessary **or** +- Determines next steps + + - Close bug as ``RESOLVED WONTFIX`` with comment as to why and + thanking submitter + - If bug is similar enough to work in roadmap, close bug as + ``RESOLVED DUPLICATE`` of the feature bug it is similar to + + - If there’s not a feature bug created already, then consider + making this bug the feature bug + + - Set type to ``enhancement`` + + - Set bug to ``P5`` priority with comment thanking submitter, and + explaining that the request will be considered for future roadmaps diff --git a/docs/bug-mgmt/policies/regressions-github.rst b/docs/bug-mgmt/policies/regressions-github.rst new file mode 100644 index 0000000000..a981e79c5e --- /dev/null +++ b/docs/bug-mgmt/policies/regressions-github.rst @@ -0,0 +1,151 @@ +Regressions from GitHub +======================= + +Release Management and the weekly regression triage must be aware of the +status of all reported regressions in order to assure we are not +shipping known regressions in Firefox releases. + +If a team is using GitHub to manage their part of the Firefox project, +there’s a risk that those groups might not see a regression. + +We need an agreed to standard for how we keep track of these. + +Policy +------ + +*All Firefox components, even if their bugs are tracked off of Bugzilla, +must have a component in Bugzilla.* + +*If a regression bug is found in any of the release trains (Nightly, +Beta, Release, or ESR) and the bug is in a Firefox component which uses +an external repository, the regression must be tracked by a bug in +Bugzilla (whether or not the component in question uses an external +issue tracker).* + +*Unless approved by Release Management, any GitHub managed code with +open regressions will not be merged to mozilla-central. Even if the +regression is not code that has been previously merged into +mozilla-central.* + +*All Firefox code managed in GitHub which uses GitHub to manage +issues* `must use the shared +tags <https://mozilla.github.io/bmo-harmony/labels>`__. + +Comments +~~~~~~~~ + +The bug **must** have the regression keyword. + +The bug **must** have release flags set. + +If the team works in an external bug tracker, then the Bugzilla bug +**must** reference, using the see-also field, the URL of the bug in the +external tracker. + +The bug **must not** be RESOLVED until the code from the external +repository containing the change set for the bug has landed in +mozilla-central. When the change set lands in mozilla-central, the +Bugzilla tracking bug should be set to RESOLVED FIXED and release status +flags should be updated to reflect the release trains the fix has been +landed or uplifted into. + +If the change set containing the patch for the regression is reverted +from mozilla-central, for any reason, then the tracking bug for the +regression **must** be set to REOPENED and the release status flags +updated accordingly. + +If the change set containing the patch for the bug is backed out, for +any reason, the bug must be reopened and the status flags on the +Bugzilla tracking bug updated. + +The team responsible for the component with the regression **should** +strive to create a patch for mozilla-central which contains the fix for +the bug alone, not a monolithic patch containing changes for several +other bugs or features. + +Landings of third-party libraries `must contain a manifest +file <https://docs.google.com/document/d/12ihxPXBo9zBBaU_pBsPrc_wNHds4Upr-PwFfiSHrbu8>`__. + +Best Practices +-------------- + +You must file a regression bug in Bugzilla +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +*If the code with the regression has landed in mozilla-central, you must +file a regression bug.* + +Example +^^^^^^^ + +While using a release of Firefox (Nightly, Beta, Release, ESR) you run +across a bug. Upon research using MozRegression or other tools you find +that the bug was introduced in a change set imported from a component +whose code and issues are managed in GitHub. + +Actions to take +''''''''''''''' + +- Open a new bug in Bugzilla in appropriate component and add the + REGRESSION keyword +- Set affected status for the releases where the bug appears +- Open an issue in the corresponding GitHub project, put the Bugzilla + bug number in the title with the prefix ‘Bug’ (i.e. “Bug 99999: + Regression in foo”) +- Add the REGRESSION label to the new issue +- Add the link to the GitHub issue into the ‘See Also” field in the + Bugzilla bug + +Consequences +'''''''''''' + +*Until the regression is fixed or backed out of the GitHub repo, the +project cannot merge code into mozilla-central* + +Example +^^^^^^^ + +You import a development branch of a component managed in GitHub into +your copy of master. You find a bug and isolate it to the imported +branch. The code is managed in their own GitHub project, but bugs are +managed in Bugzilla. + +Actions to take +''''''''''''''' + +- Open a new bug in Bugzilla in appropriate component and add the + REGRESSION keyword +- Set affected status for the releases where the bug appears + +Consequences +'''''''''''' + +*Until the regression is fixed or backed out of the GitHub repo, the +project cannot merge code into mozilla-central* + +Do not file a regression bug in Bugzilla +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +*If the code with the regression has not landed in mozilla-central, you +do not need to file a bug.* + + +Example +^^^^^^^ + +You import a development branch of a component managed in GitHub into +your copy of master. You find a bug and isolate it to the imported +branch. The code and issues are managed in their own GitHub project. + + +Actions to take +''''''''''''''' + +- File new issue in the GitHub repository of the imported code. +- Label issue as REGRESSION + +Consequence +''''''''''' + +*Issue blocks merge of GitHub project with mozilla-central until +resolved or backed out.* diff --git a/docs/bug-mgmt/policies/triage-bugzilla.rst b/docs/bug-mgmt/policies/triage-bugzilla.rst new file mode 100644 index 0000000000..53e26b31ca --- /dev/null +++ b/docs/bug-mgmt/policies/triage-bugzilla.rst @@ -0,0 +1,277 @@ +Triage for Bugzilla +=================== + +Expectations +------------ + +All teams working on Firefox using either or both Mozilla-central and +Bugzilla are expected to follow the following process. + +What is a Triaged Bug +--------------------- + +The new definition of Triaged will be Firefox-related bugs of type +``defect`` where the component is not +``UNTRIAGED``, and a :ref:`Severity <Defect Severity>` value not equal +to ``--`` or ``N/A``. + +Bugs of type Task or Enhancement may have a Severity of ``N/A``, +but defects must have a Severity that is neither ``--`` nor +``N/A``. + +Why Triage +---------- + +We want to make sure that we looked at every defect and a severity has +been defined. This way, we make sure that we did not miss any critical +issues during the development and stabilization cycles. + +Staying on top of the bugs in your component means: + +- You get ahead of critical regressions and crashes which could trigger + a point release if uncaught + + - And you don’t want to spend your holiday with the Release + Management team (not that they don’t like you) + +- Your bug queue is not overwhelming + + - Members of your team do not see the bug queue and get the + ‘wiggins’ + +Who Triages +----------- + +Engineering managers and directors are responsible for naming the +individuals responsible for triaging :ref:`all types of bugs <Bug Types>` in a component. + +We use Bugzilla to manage this. See the `list of triage +owners <https://bugzilla.mozilla.org/page.cgi?id=triage_owners.html>`__. + +If you need to change who is responsible for triaging a bug in a +component, please `file a bug against bugzilla.mozilla.org in the +Administration +component <https://bugzilla.mozilla.org/enter_bug.cgi?product=bugzilla.mozilla.org&component=Administration>`__. +When a new component is created, a triage owner **must** be named. + +Rotating triage +~~~~~~~~~~~~~~~ + +Some components are monitored by a rotation of triagers. In those cases, +the triage owner on Bugzilla will be automatically updated to reflect the +person on the rotation. The rotations are managed as calendars. + +If you wish to set up a rotation for triaging one or more components, +add a link to your rotation calendar in the `triage rotations spreadsheet <https://docs.google.com/spreadsheets/d/1EK6iCtdD8KP4UflIHscuZo6W5er2vy_TX7vsmaaBVd4>`__. + +Firefox::General and Toolkit::General +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Bugs in Firefox::General are fitted with Bug Bug’s model to see if +there’s another component with a high likelihood of fit, and if a +threshold confidence is achieved, the bug is moved to that component. + +Members of the community also review bugs in this component and try to +move them. + +What Do You Triage +------------------ + +As a triage owner the queries you should be following for your component +are: + +- All open bugs, in your components without a pending ``needinfo`` flag + which do not have a valid value of severity set +- All bugs with active review requests in your component which have not + been modified in five days +- All bugs with reviewed, but unlanded patches in your components +- All bugs with a needinfo request unanswered for more than 10 days + +There’s a tool with these queries to help you find bugs +https://bugdash.moz.tools/ and the source is at +https://github.com/mozilla/bugdash/. + +If a bug is an enhancement it needs a priority set and a target release +or program milestone. These bugs are normally reviewed by product +managers. Enhancements can lead to release notes and QA needed that we +also need to know about + +If a bug is a task resulting in a changeset, release managers will need +to known when this work will be done. A task such as refactoring fragile +code can be risky. + +Weekly or More Frequently (depending on the component) find un-triaged +bugs in the components you triage. + +Decide the :ref:`Severity <Defect Severity>` for each untriaged bug +(you can override what’s already been set.) + +These bugs are reviewed in the weekly Regression Triage meeting + +- Bugs of type ``defect`` with the ``regression`` keyword without + ``status-firefoxNN`` decisions +- Bugs of type ``defect`` with the ``regression`` keyword without + a regression range + +Automatic Bug Updates +~~~~~~~~~~~~~~~~~~~~~ + +When a bug is tracked for a release, i.e. the ``tracking_firefoxNN`` +flag is set to ``+`` or ``blocking`` triage decisions will be overridden, +or made as follows: + +- If a bug is tracked for or blocking beta, release or ESR, its + priority will be set to ``P1`` +- If a bug is tracked for or blocking nightly, its priority will be set + to ``P2`` + +Because bugs can be bumped in priority it’s essential that triage owners +review their +`P1 <https://bugzilla.mozilla.org/buglist.cgi?priority=P1&f1=triage_owner&o1=equals&resolution=---&v1=%25user%25>`__ +and +`P2 <https://bugzilla.mozilla.org/buglist.cgi?priority=P2&f1=triage_owner&o1=equals&resolution=---&v1=%25user%25>`__ +bugs frequently. + +Assumptions +~~~~~~~~~~~ + +If a bug's release status in Firefox version N was ``affected`` or ``wontfix``, +its Severity is ``S3`` or ``S4`` and its Priority is ``P3`` or lower (backlog,) +then its release status in Firefox version N+1, if the bug is still open, +is considered to be ``wontfix``. + +Questions and Edge Cases +------------------------ + +This bug is a feature request +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Set the bug’s type to ``enhancement``, add the ``feature`` keyword if +relevant, and state to ``NEW``. Set the bug's Severity to ``N/A``. This +bug will be excluded from future triage queries. + +This bug is a task, not a defect +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Set the bug’s type to ``task``, and state to ``NEW``. Set the bug's +Severity to ``N/A``. This bug will be excluded from future triage queries. + + +If you are not sure of a bug’s type, check :ref:`our rules for bug +types <Bug Types>`. + +This bug’s state is ``UNCONFIRMED`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Are there steps to reproduce? If not, needinfo the person who filed the +bug, requesting steps to reproduce. You are not obligated to wait +forever for a response, and bugs for which open requests for information +go unanswered can be ``RESOLVED`` as ``INCOMPLETE``. + +I need help reproducing the bug +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Set a needinfo for the QA managers, Softvision project managers, or the +QA owner of the component of the bug. + +I don’t have enough information to make a decision +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If you don’t have a reproduction or confirmation, or have questions +about how to proceed, ``needinfo`` the person who filed the bug, or +someone who can answer. + +The ``stalled`` keyword +~~~~~~~~~~~~~~~~~~~~~~~ + +The extreme case of not-enough-information is one which cannot be +answered with a ``needinfo`` request. The reporter has shared all they +know about the bug, we are out of strategies to take to resolve it, but +the bug should be kept open. + +Mark the bug as stalled by adding the ``stalled`` keyword to it. The +keyword will remove it from the list of bugs to be triaged. + +If a patch lands on a ``stalled`` bug, automation will remove the +keyword. Otherwise, when the ``keyword`` is removed, the bug will have +its priority reset to ``--`` and the components triage owner notified by +automation. + +Bugs which remain ``stalled`` for long periods of time should be +reviewed, and closed if necessary. + +Bug is in the wrong Component +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If the bug has a Severity of ``S3``, ``S4``, or ``N/A`` move the what +you think is the correct component, or needinfo the person +responsible for the component to ask them. + +If the bug has a Severity of ``S1`` or ``S2`` then notify Release Management +and contact the triage owner of the component for which you think it belongs to. +We cannot lose track of a high severity bug because it is in the wrong component. + +My project is on GitHub +~~~~~~~~~~~~~~~~~~~~~~~ + +We have :ref:`a guide for GitHub projects to follow <GitHub Metadata Recommendations>` when +triaging. (Note: this guide needs updating.) + +Summary +------- + +Multiple times weekly +~~~~~~~~~~~~~~~~~~~~~ + +Use queries for the components you are responsible for in +https://github.com/mozilla/bugdash/ to find bugs in +need of triage. + +For each untriaged bug: + +- Assign a Severity +- **Do not** assign a ``defect`` a Severity of + ``N/A`` + +You can, but are not required to set the bug's :ref:`Priority <Priority Definitions>`. + +Watch open needinfo flags +~~~~~~~~~~~~~~~~~~~~~~~~~ + +Don’t let open needinfo flags linger for more than two weeks. + +Close minor bugs with unresponded needinfo flags. + +Follow up on needinfo flag requests. + +`BugDash <https://github.com/mozilla/bugdash/>`__ will help you find these. + +End of Iteration/Release Cycle +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Any open ``S1`` or ``S2`` bugs at the end of the release cycle +will require review by engineering and release management. A +policy on this is forthcoming. + +Optional +^^^^^^^^ + +(The guidelines on bug priority are under review.) + +Are there open P1s? Revisit their priority, +and move to them to the backlog (``P3``) or ``P2``. + +Are there ``P2`` bugs that should move to ``P1`` +for the next cycle? + +Are there ``P2`` you now know are lower priority, +move to ``P3``. + +Are there ``P3`` bugs you now know you won’t get to? +Either demote to ``P5`` (will accept patch) or +resolve as ``WONTFIX``. + +Getting help +------------ + +- Ask in #bug-handling on chat.mozilla.org diff --git a/docs/bug-mgmt/processes/accessibility-review.md b/docs/bug-mgmt/processes/accessibility-review.md new file mode 100644 index 0000000000..3358defd7e --- /dev/null +++ b/docs/bug-mgmt/processes/accessibility-review.md @@ -0,0 +1,100 @@ +# Accessibility Review + +## Introduction +At Mozilla, accessibility is a fundamental part of our mission to ensure the +internet is "open and accessible to all," helping to empower people, regardless +of their abilities, to contribute to the common good. Accessibility Review is a +service provided by the Mozilla Accessibility team to review features and +changes to ensure they are accessible to and inclusive of people with +disabilities. + +## Do I Need Accessibility Review? +You should consider requesting accessibility review if you aren't certain +whether your change is accessible to people with disabilities. Accessibility +review is optional, but it is strongly encouraged if you are introducing new +user interface or are significantly redesigning existing user interface. +Review should be requested both on the design side _and_ on the engineering side. + +## When Should I Request Accessibility Review? +Generally, it's best to request accessibility review as early as possible, even +during the product requirements or UI design stage. Particularly for more +complex user interfaces, accessibility is much easier when incorporated into the +design, rather than attempting to retro-fit accessibility after the +implementation is well underway. + +The accessibility team has developed the [Mozilla Accessibility Release +Guidelines](https://wiki.mozilla.org/Accessibility/Guidelines) which outline +what is needed to make user interfaces accessible. To make accessibility review +faster, you may wish to try to verify and implement these guidelines prior to +requesting accessibility review. + +For design reviews, please allow at least a week between review request and expected-engineering-handoff. The deadline for engineering review requests is Friday of the first week of nightly builds for the release in which the feature/change is expected to ship. +This is the same date as the PI Request deadline. + +## Requesting Design Review +Design review should be requested using the Accessibility Review Request +workflow in the #accessibility slack channel. To access this workflow, click on +the workflow link in the #accessibility slack channel header or in the links +sidebar. You can also type `/accessibility design review request` from any +channel or DM. Then, fill in the information requested. + +Please complete the following self-review tasks **before requesting review**. Note: Some of the following links require SSO authentication. + +- **Perform a contrast audit**: Using a [figma plugin that audits contrast](https://www.figma.com/community/plugin/748533339900865323/Contrast), check your designs for color contrast sufficiency. Your designs should be at least "AA" rated in order to pass accessibility review. "AAA" is even better! If there are particular components that are difficult to adjust to meet "AA" standards, make a note in the figma file and the a11y team will provide specific guidance during review. +- **Add focus order and role annotations**: Focus order annotations describe the behaviour a keyboard user should expect when navigating your design. Generally this follows the reading order. Note that we only care about focusable elements here (ie. links, buttons, inputs, etc.). +Role annotations help screen readers and other assistive technologies identify the "kind" of component they're navigating through. These mappings expose semantic information to the user. You can find a [list of common roles here](https://www.codeinwp.com/blog/wai-aria-roles/). You may want to use a [figma plugin that annotates focus and roles](https://www.figma.com/community/plugin/731310036968334777/A11y---Focus-Order) for this process. You do not need to annotate every view in your design, pick those with the largest amount of new content. +- **Create Windows High-Contrast Mode (HCM) mockups**: Our designs should be accessible to users [running HCM](https://docs.google.com/document/d/1El3XJiAdA5gFcG7H9iI1dNmLbht0hXfi_oKBZPWx3t0/edit). You can read more about [how HCM affects color selection](https://firefox-source-docs.mozilla.org/accessible/ColorsAndHighContrastMode.html), and [how to design for HCM](https://wiki.mozilla.org/Accessibility/Design_Guide). Using the ["Night Sky" HCM palette](https://www.figma.com/file/XQrEePCCJebjlVBQwNggQ6/Pro-Client-Accessibility-Reviews?node-id=25%3A3848), translate your designs into High Contrast. Remember, it's important we use these colors **semantically**, not based on a desire for a particular aesthetic. Colors are labelled according to their uses -- `Background` for page background, `Button Text` for button or control text, `Selected Item Background` for backgrounds of selected or active items, etc.. You do not need to mock up every view in your design, pick those with the largest amount of new content. You can find [examples of previous HCM mock ups here](https://www.figma.com/file/XQrEePCCJebjlVBQwNggQ6/Accessibility). +Where possible, we should rely on SVG's and PNG's for image content to increase adaptability. + +## Requesting Engineering Review +For an engineering-focused review, you submit a review request by setting the a11y-review flag to "requested" +on a bug in Bugzilla and filling in the template that appears in the comment +field. For features spanning several bugs, you may wish to file a new, dedicated +bug for the accessibility review. Otherwise, particularly for smaller changes, +you may do this on an existing bug. Note that if you file a new bug, you will +need to submit the bug and then edit it to set the flag. + +Once the review is done, the accessibility team will set the a11y-review +flag depending on the outcome: + +- passed: Either no changes are required, or some changes are required but the + accessibility team does not believe it is necessary to review or verify + those changes prior to shipping. Generally, a review will not be passed if + there are outstanding s2 or certain high impact s3 accessibility defects + which should block the feature or change from shipping. However, despite + their high severity, some s2 defects are trivial to fix (e.g. missing + accessibility labels), so the accessibility team may elect to pass the + review on the condition that these are fixed promptly. +- changes required: Changes are required and the accessibility team will need to + review or verify those changes before determining whether the accessibility + of the feature or change is acceptable for shipping. It is the + responsibility of the requesting engineering team to re-request + accessibility review once changes are made to address the accessibility + team's initial round of feedback. + +## Phabricator Review Group +There is also an +[accessibility-frontend-reviewers](https://phabricator.services.mozilla.com/tag/accessibility-frontend-reviewers/) +group on Phabricator. This should generally only be used if: + +1. The patch is a change requested as part of an accessibility review as + described above; or +2. There is a very specific aspect of accessibility implemented in the patch and + you would like the accessibility team to confirm whether it has been + implemented correctly. + +If you would instead like the accessibility team to assess **whether a feature +or change is sufficiently accessible**, please request an accessibility +**engineering** review as described above. Without this, the accessibility team +will not have sufficient context or understanding of the change or how to test +it, which is necessary to thoroughly assess its accessibility. + +## Questions? +If you have any questions, please don't hesitate to contact the Accessibility +team: + +* \#accessibility on + [Matrix](https://matrix.to/#/!jmuErVonajdNMbgdeY:mozilla.org?via=mozilla.org&via=matrix.org) + or [Slack](https://mozilla.slack.com/archives/C4E0W8B8E) +* Email: accessibility@mozilla.com +* Please avoid reaching out to individual team members directly -- containing review requests and questions in these channels helps us balance our workload. Thank you! diff --git a/docs/bug-mgmt/processes/doc-requests.rst b/docs/bug-mgmt/processes/doc-requests.rst new file mode 100644 index 0000000000..dc390a27cb --- /dev/null +++ b/docs/bug-mgmt/processes/doc-requests.rst @@ -0,0 +1,39 @@ +User documentation requests +=========================== + +If you are working on a change (bugfix, enhancement, or feature) which +would benefit from user-facing documentation, please use the +``user-doc-firefox`` flag to request it. + +This flag can be modified by anyone with ``EDITBUGS`` privileges. + +The default value of the flag is ``---``. + +If the bug needs user-facing documentation, set the flag to +``docs-needed``. This flag will be monitored by the support.mozilla.org +(SUMO) team. + +Once the docs are ready to be published, set the flag to +``docs-completed``. + +If it’s determined that documentation is not need after setting the flag +to ``docs-needed``, update the flag to ``none-needed`` so we know that +it’s been reviewed. + +Summary +------- + +=========== == ============== +From To +=========== == ============== +— to none-needed +— to docs-needed +docs-needed to none-needed +docs-needed to docs-completed +=========== == ============== + +Notes +----- + +A flag is used instead of the old keywords because flags can be +restricted to a subset of products and components. diff --git a/docs/bug-mgmt/processes/fixing-security-bugs.rst b/docs/bug-mgmt/processes/fixing-security-bugs.rst new file mode 100644 index 0000000000..e07853ac79 --- /dev/null +++ b/docs/bug-mgmt/processes/fixing-security-bugs.rst @@ -0,0 +1,217 @@ +Fixing Security Bugs +==================== + +A bug has been reported as security-sensitive in Bugzilla and received a +security rating. + +If this bug is private - which is most likely for a reported security +bug - **the process for patching is slightly different than the usual +process for fixing a bug**. + +Here are security guidelines to follow if you’re involved in reviewing, +testing and landing a security patch. See +:ref:`Security Bug Approval Process` +for more details about how to request sec-approval and land the patch. + +Keeping private information private +----------------------------------- + +A security-sensitive bug in Bugzilla means that all information about +the bug except its ID number are hidden. This includes the title, +comments, reporter, assignee and CC’d people. + +A security-sensitive bug usually remains private until a fix is shipped +in a new release, **and after a certain amount of time to ensure that a +maximum number of users updated their version of Firefox**. Bugs are +usually made public after 6 months and a couple of releases. + +From the moment a security bug has been privately reported to the moment +a fix is shipped and the bug is set public, all information about that +bug needs to be handled carefully in order to avoid an unmitigated +vulnerability becoming known and exploited before we release a +fix (0-day). + +During a normal process, information about the nature of bug can be +accessed through: + +- Bug comments (Bugzilla, GitHub issue) +- Commit message (visible on Bugzilla, tree check-ins and test servers) +- Code comments +- Test cases +- Bug content can potentially be discussed on public IRC/Slack channels + and mailing list emails. + +When patching for a security bug, you’ll need to be mindful about what +type of information you share and where. + +In commit messages +~~~~~~~~~~~~~~~~~~ + +People are watching code check-ins, so we want to avoid sharing +information which would disclose or help finding a vulnerability too +easily before we shipped the fix to our users. This includes: + +- The **nature of the vulnerability** (overflow, use-after-free, XSS, + CSP bypass...) +- **Ways to trigger and exploit that vulnerability** + - In commit messages, code comments and test cases. +- The fact that a bug / commit is security-related: + + - **Trigger words** in the commit message or code comments such as + "security", "exploitable", or the nature of a security vulnerability + (overflow, use-after-free…) + - **Security approver’s name** in a commit message. +- The Firefox versions and components affected by the vulnerability. +- Patches with an obvious fix. + +In Bugzilla and other public channels +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In addition to commits, you’ll need to be mindful of not disclosing +sensitive information about the bug in public places, such as Bugzilla: + +- Mention the bugs in comment of the private bug instead. +- Do not comment sensitive information in public related bugs. +- Also be careful about who you give bug access to: **double check + before CC’ing the wrong person or alias**. +- As of recently, you may now add public bugs in the “duplicate”, + “depends on”, “blocks”, “regression”, “regressed by”, or “see also” section. + Bugzilla will only reveal those relationships to people with ``editbugs`` + permission or access to the security bug. + +On IRC, Slack channels, GitHub issues, mailing lists: If you need to +discuss about a security bug, use a private channel (protected with a +password or with proper right access management) + +During Development +------------------ + +Testing security bugs +~~~~~~~~~~~~~~~~~~~~~ + +Pushing to Try servers requires Level 1 Commit access but **content +viewing is publicly accessible**. + +As much as possible, **do not push to Try servers**. Testing should be +done locally before checkin in order to prevent public disclosing of the +bug. + +Because of the public visibility, pushing to Try has all the same concerns +as committing the patch. Please heed the concerns in the +:ref:`landing-your-patch` section before thinking about it, and check with +the security team for an informal "sec-approval" before doing so. + +**Do not push the bug's own vulnerability testcase to Try.** + +If you need to push to Try servers, make sure your tests don’t disclose +what the vulnerability is about or how to trigger it. Do not mention +anywhere it is security related. + +Obfuscating a security patch +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If your security patch looks obvious because of the code it contains +(e.g. a one-line fix), or if you really need to push to Try servers, +**consider integrating your security-related patch to non-security work +in the same area**. And/or pretend it is related to something else, like +some performance improvement or a correctness fix. **Definitely don't +include the bug number in the commit message.** This will help making +the security issue less easily identifiable. (The absolute ban against +"Security through Obscurity" is in relation to cryptographic systems. In +other situations you still can't *rely* on obscurity but it can +sometimes buy you a little time. In this context we need to get the +fixes into the hands of our users faster than attackers can weaponize +and deploy attacks and a little extra time can help.) + +Requesting sec-approval +~~~~~~~~~~~~~~~~~~~~~~~ + +See :ref:`Security Bug Approval Process` +for more details + +.. _landing-your-patch: + +Landing your patch (with or without sec-approval) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Before asking for sec-approval or landing, ensure your patch does not disclose +information about the security vulnerability unnecessarily. Specifically: + +#. The patch commit message and its contents should not mention security, + security bugs, or sec-approvers. + Note that you can alter the commit message directly in phabricator, + if that's the only thing you need to do - you don't need to amend your + local commit and re-push it. + While comprehensive commit messages are generally encouraged; they + should be omitted for security bugs and instead be posted in the bug + (which will eventually become public.) +#. Separate out tests into a separate commit. + **Do not land tests when landing the patch. Remember we don’t want + to 0-day ourselves!** This includes when pushing to try. + + - Tests should only be checked in later, after an official Firefox + release that contains the fix has been live for at least + four weeks. For example, if Firefox 53 + contains a security issue that affects the world and that issue is + fixed in 54, tests for this fix should not be checked in + until four weeks after 54 goes live. + + The exception to this is if there is a security issue that doesn't + affect any release branches, only mozilla-central and/or other + development branches. Since the security problem was never + released to the world, once the bug is fixed in all affected + places, tests can be checked in to the various branches. + - There are two main techniques for remembering to check in the + tests later: + + a. clone the sec bug into a separate "task" bug **that is also + in a security-sensitive group to ensure it's not publicly visible** + called something like "land tests for bug xxxxx" and assign to + yourself. It should get a "sec-other" keyword rating. + + Tip: In phabricator, you can change the bug linked to + a commit with tests if the tests were already separate, while keeping + the previously granted review, meaning you can just land the patch + when ready, rather than having your reviewer and you have to remember + what this was about a month or two down the line. + b. Or, set the "in-testsuite" flag to "?", and later set it to "+" + when the tests get checked in. + + +Landing tests +~~~~~~~~~~~~~ + +Tests can be landed **once the release containing fixes has been live +at least 4 weeks**. + +The exception is if a security issue has never been shipped in a release +build and has been fixed in all development branches. + +Making a security bug public +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This is the responsibility of the security management team. + +Essentials +---------- + +- **Do not disclose any information about the vulnerability before a + release with a fix has gone live for enough time for users to update + their software**. + + - This includes code comments, commit messages, tests, public + communication channels. + +- If any doubt: '''request sec-approval? ''' +- If any doubt: **needinfo security folks**. +- **If there’s no rating, assume the worst and treat the bug as + sec-critical**. + +Documentation & Contacts +------------------------ + +- :ref:`Normal process for submitting a patch <How to submit a patch>` +- `How to file a security bug <https://wiki.mozilla.org/Security/Fileabug>`__ +- `Handling Mozilla security bugs (policy) <https://www.mozilla.org/en-US/about/governance/policies/security-group/bugs/>`__ +- :ref:`Security Bug Approval Process` +- `Contacting the Security team(s) at Mozilla: <https://wiki.mozilla.org/Security>`__ diff --git a/docs/bug-mgmt/processes/labels.rst b/docs/bug-mgmt/processes/labels.rst new file mode 100644 index 0000000000..60c1bbeb2b --- /dev/null +++ b/docs/bug-mgmt/processes/labels.rst @@ -0,0 +1,155 @@ +GitHub Metadata Recommendations +=============================== + +To have better consistency with code and task tracking among Mozilla +Central, Bugzilla, and GitHub, we request that you use a common set of +labels in your projects. Benefits of improved consistency in our +conventions include: + +- Consistency makes measurement of processes simpler across the + organization +- Consistency makes it easier to write reusable process tools +- Consistency increases clarity for those than need to work across + different repositories and bug trackers +- Consistency reduces friction around engineering mobility between + projects + +We recommend creating sets of labels in your project to do this. + +Bug types +--------- + +In Bugzilla bugs are distinguished by type: ``defect``, ``enhancement``, +and ``tasks``. Use a label to make this distinction in your project. + +Statuses +-------- + +Bugs in GitHub issues have two states: closed and open. Bugzilla has a +richer set of states. + +When you close a bug, add a label indicating `the +resolution <https://wiki.mozilla.org/BMO/UserGuide/BugStatuses#Resolutions>`__. + +- ``fixed`` + + - A change set for the bug has been landed in Mozilla-Central + - A GitHub issue could be closed, but the change set has not + landed so it would be still considered open from the + Bugzilla point of view + +- ``invalid`` + + - The problem described is not a bug. + +- ``incomplete`` + + - The problem is vaguely described with no steps to reproduce, or is + a support request. + +- ``wontfix`` + + - The problem described is a bug which will never be fixed. + +- ``duplicate`` + + - The problem is a duplicate of an existing bug. Be sure to link the + bug this is a duplicate of. + +- ``worksforme`` + + - All attempts at reproducing this bug were futile, and reading the + code produces no clues as to why the described behavior would + occur. + +Severities (Required) +--------------------- + +The triage process for Firefox bugs in Bugzilla requires a non default +value of a bug's :ref:`Severity (definitions) <Defect Severity>`. + +Release Status Flags +-------------------- + +Open Firefox bugs may also have :ref:`status flags <Release Status Flags>` +(``status_firefoxNN``) set for Nightly, Beta, Release, or ESR. + +Priorities +---------- + +Firefox projects in Bugzilla can use the :ref:`priority field <Priority Definitions>` +to indicate when a bug will be worked on. + +Keywords +-------- + +In GitHub issues metadata is either a label or the bug’s open/closed +state. + +Some Bugzilla metadata behaves like labels, but you need to be careful +with how you use it in order not to confuse QA. + +Regressions +~~~~~~~~~~~ + +In Bugzilla, the ``regression`` keyword indicates a regression in +existing behavior introduced by a code change. + +When a bug is labeled as a regression in GitHub does it imply the +regression is in the code module in GitHub, or the module that’s landed +in Mozilla Central? Using the label ``regression-internal`` will signal +QA that the regression is internal to your development cycle, and not +one introduced into the Mozilla Central tree. + +If it is not clear which pull request caused the regression, add the +``regressionwindow-wanted`` label. + +Other Keywords +~~~~~~~~~~~~~~ + +Other useful labels include ``enhancement`` to distinguish feature +requests, and ``good first issue`` to signal to contributors (`along +with adequate +documentation <http://blog.humphd.org/why-good-first-bugs-often-arent/>`__.) + +Summary +------- + +To represent Bugzilla fields, use labels following this scheme. + +- Bug types + + - ``defect``, ``enhancement``, ``task`` + +- Resolution statuses + + - ``invalid``, ``duplicate``, ``incomplete``, ``worksforme``, + ``wontfix`` + +- Regressions + + - ``regression``, ``regressionwindow-wanted``, + ``regression-internal`` + + +- :ref:`Severity <Defect Severity>` (required) + + - ``S1``, ``S2``, ``S3``, ``S4``, ``N/A`` (reserved for bugs + of type ``task`` or ``enhancement``) + +- :ref:`Status flags <Release Status Flags>` + + - ``status_firefoxNN:<status>`` + (example ``status_firefox77:affected``) + +- :ref:`Priority <Priority Definitions>` + + - ``P1``, ``P2``, ``P3``, ``P5`` + +- Other keywords + + - ``good first bug``, ``perf``, &etc. + + +You may already have a set of tags, so do an edit to convert them +or use `the GitHub settings app <https://github.com/probot/settings>`__. diff --git a/docs/bug-mgmt/processes/regressions.rst b/docs/bug-mgmt/processes/regressions.rst new file mode 100644 index 0000000000..e603902623 --- /dev/null +++ b/docs/bug-mgmt/processes/regressions.rst @@ -0,0 +1,64 @@ +How to Mark Regressions +======================= + +Regressions +----------- + +For regression bugs in Mozilla-Central, our policy is to tag the bug as +a regression, identify the commits which caused the regression, then +mark the bugs associated with those commits as causing the regression. + +What is a regression? +--------------------- + +A regression is a bug (in our scheme a ``defect``) introduced by a +`changeset <https://en.wikipedia.org/wiki/Changeset>`__. + +- Bug 101 *fixes* Bug 100 with Change Set A +- Bug 102 *reported which describes previously correct behavior now not + happening* +- Bug 102 *investigated and found to be introduced by Change Set A* + +Marking a Regression Bug +------------------------ + +These things are true about regressions: + +- **Bug Type** is ``defect`` +- **Keywords** include ``regression`` +- **Status_FirefoxNN** is ``affected`` for each version (in current + nightly, beta, and release) of Firefox in which the bug was found +- The bug’s description covers previously working behavior which is no + longer working + +Until the change set which caused the regression has been found through +`mozregression <https://mozilla.github.io/mozregression/>`__ or another +bisection tool, the bug should also have the ``regressionwindow-wanted`` +keyword. + +Once the change set which caused the regression has been identified, +remove the ``regressionwindow-wanted`` keyword and set the **Regressed +By** field to the id of the bug associated with the change set. + +Setting the **Regressed By** field will update the **Regresses** field +in the other bug. + +Set a needinfo for the author of the regressing patch asking them to fix +or revert the regression. + +Previous Method +--------------- + +Previously we over-loaded the **Blocks** and **Blocked By** fields to +track the regression, setting **Blocks** to the id of the bug associated +with the change set causing the regression, and using the +``regression``, ``regressionwindow-wanted`` keywords and the status +flags as described above. + +This made it difficult to understand what was a dependency and what was +a regression when looking at dependency trees in Bugzilla. + +FAQs +---- + +*To be written* diff --git a/docs/bug-mgmt/processes/security-approval.rst b/docs/bug-mgmt/processes/security-approval.rst new file mode 100644 index 0000000000..a4c29730e7 --- /dev/null +++ b/docs/bug-mgmt/processes/security-approval.rst @@ -0,0 +1,192 @@ +Security Bug Approval Process +============================= + +How to fix a core-security bug in Firefox - developer guidelines +---------------------------------------------------------------- + +Follow these security guidelines if you’re involved in reviewing, +testing and landing a security patch: +:ref:`Fixing Security Bugs`. + +Purpose: don't 0-day ourselves +------------------------------ + +People watch our check-ins. They may be able to start exploiting our +users before we can get an update out to them if + +- the patch is an obvious security fix (bounds check, kungFuDeathGrip, + etc.) +- the check-in comment says "security fix" or includes trigger words + like "exploitable", "vulnerable", "overflow", "injection", "use after + free", etc. +- comments in the code mention those types of things or how someone + could abuse the bug +- the check-in contains testcases that show exactly how to trigger the + vulnerability + +Principle: assume the worst +--------------------------- + +- If there's no rating we assume it could be critical +- If we don't know the regression range we assume it needs porting to + all supported branches + +Process for Security Bugs (Developer Perspective) +------------------------------------------------- + +Filing / Managing Bugs +~~~~~~~~~~~~~~~~~~~~~~ + +- Try whenever possible to file security bugs marked as such when + filing, instead of filing them as open bugs and then closing later. + This is not always possible, but attention to this, especially when + filing from crash-stats, is helpful. +- It is _ok_ to link security bugs to non-security bugs with Blocks, + Depends, Regressions, or See Also. Users with the editbugs permission + will be able to see the reference, but not view a restricted bug or + its summary. Users without the permission will not be able to see the link. + For critical severity bugs where even that seems problematic, consider + mentioning the bug in a comment on the security bug instead. We can always + fill in the links later after the fix has shipped. + +Developing the Patch +~~~~~~~~~~~~~~~~~~~~ + +- Comments in the code should not mention a security issue is being + fixed. Don’t paint a picture or an arrow pointing to security issues + any more than the code changes already do. +- Do not push to Try servers if possible: this exposes the security + issues for these critical and high rated bugs to public viewing. In + an ideal case, testing of patches is done locally before final + check-in to mozilla-central. +- If pushing to Try servers is necessary, **do not include the bug + number in the patch**. Ideally, do not include tests in the push as + the tests can illustrate the exact nature of the security problem + frequently. +- If you must push to Try servers, with or without tests, try to + obfuscate what this patch is for. Try to push it with other, + non-security work, in the same area. + +Request review of the patch in the same process as normal. After the +patch has been reviewed you will request sec-approval as needed. See +:ref:`Fixing Security Bugs` +for more examples/details of these points. + +Preparing the patch for landing +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +See :ref:`Fixing Security Bugs` +for more details. + +On Requesting sec-approval +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +For security bugs with no sec- severity rating assume the worst and +follow the rules for sec-critical. During the sec-approval process we +will notice it has not been rated and rate it during the process. + +Core-security bug fixes can be landed by a developer without any +explicit approval if: + +| **A)** The bug has a sec-low, sec-moderate, sec-other, or sec-want + rating. +| **or** +| **B)** The bug is a recent regression on mozilla-central. This means + +- A specific regressing check-in has been identified +- The developer can (**and has**) marked the status flags for ESR and + Beta as "unaffected" +- We have not shipped this vulnerability in anything other than a + nightly build + +If it meets the above criteria, developers do not need to ask for sec-approval. + +In all other cases, developers should ask for sec-approval. +Set the sec-approval flag to '?' on the patch when it is ready to be landed. +You will find these flags in Bugzilla using the "Details" links in the +Bugzilla attachment table (not directly on phabricator at time of writing). + +If developers are unsure about a bug and it has a patch ready, just +request sec-approval anyway and move on. Don't overthink it! + +An automatic nomination comment will be added to bugzilla when +sec-approval is set to '?'. The questions in this need to be filled out +as best as possible when sec-approval is requested for the patch. + +It is as follows (courtesy of Dan Veditz):: + + [Security approval request comment] + How easily can the security issue be deduced from the patch? + Do comments in the patch, the check-in comment, or tests included in + the patch paint a bulls-eye on the security problem? + Which older supported branches are affected by this flaw? + If not all supported branches, which bug introduced the flaw? + Do you have backports for the affected branches? If not, how + different, hard to create, and risky will they be? + How likely is this patch to cause regressions; how much testing does + it need? + +This is similar to the ESR approval nomination form and is meant to help +us evaluate the risks around approving the patch for checkin. + +When the bug is approved for landing, the sec-approval flag will be set +to '+' with a comment from the approver to land the patch. At that +point, land it according to instructions provided.. + +This will allow us to control when we can land security bugs without +exposing them too early and to make sure they get landed on the various +branches. + +If you have any questions or are unsure about anything in this document +contact us on Slack in the #security channel or the current +sec-approvers Dan Veditz and Tom Ritter. + +Process for Security Bugs (sec-approver Perspective) +---------------------------------------------------- + +The security assurance team and release management will have their own +process for approving bugs: + +#. The Security assurance team goes through sec-approval ? bugs daily + and approves low risk fixes for central (if early in cycle). + Developers can also ping the Security Assurance Team (specifically + Tom Ritter & Dan Veditz) in #security on Slack when important. + + #. If a bug lacks a security-rating one should be assigned - possibly + in coordination with the (other member of) the Security Assurance + Team + +#. Security team marks tracking flags to ? for all affected versions + when approved for central. (This allows release management to decide + whether to uplift to branches just like always.) +#. Weekly security/release management triage meeting goes through + sec-approval + and ? bugs where beta and ESR is affected, ? bugs with + higher risk (sec-high and sec-critical), or ? bugs near end of cycle. + +Options for sec-approval including a logical combination of the +following: + +- Separate out the test and comments in the code into a followup commit + we will commit later. +- Remove the commit message and place it in the bug or comments in a + followup commit. +- Please land it bundled in with another commit +- Land today +- Land today, land the tests after +- Land closer to the release date +- Land in Nightly to assess stability +- Land today and request uplift to all branches +- Request uplift to all branches and we'll land as close to shipping as + permitted +- Chemspill time + +The decision process for which of these to choose is perceived risk on +multiple axes: + +- ease of exploitation +- reverse engineering risk +- stability risk + +The most common choice is: not much stability risk, not an immediate +reverse engineering risk, moderate to high difficulty of exploitation: +"land whenever". diff --git a/docs/bug-mgmt/processes/shared-bug-queues.rst b/docs/bug-mgmt/processes/shared-bug-queues.rst new file mode 100644 index 0000000000..dc2df9bbf9 --- /dev/null +++ b/docs/bug-mgmt/processes/shared-bug-queues.rst @@ -0,0 +1,34 @@ +Shared Bug Queues +================= + +Reviewers for change sets can be suggested at the product and component +level, but only the person who has been asked to review code will be +notified. + +Realizing that Bugzilla users can *watch* other users, `Chris +Cooper <https://mozillians.org/en-US/u/coop/>`__ came up with the idea +of having `a shared reviews alias for review +requests <http://coopcoopbware.tumblr.com/post/170952242320/experiments-in-productivity-the-shared-bug-queue>`__. + +If you want to watch a particular part of the tree in Mozilla Central, +then `use the Herald +tool <https://phabricator.services.mozilla.com/book/phabricator/article/herald/>`__. + +Process +------- + +1. Create a new bugzilla.mozilla.com account for an address which can + receive mail. + Use the ``name+extension@domain.tld`` trick such as + ``jmozillian+reviews@mozilla.com`` to create a unique address +2. Respond to the email sent by Bugzilla and set a password on the + account +3. `Open a bug <https://mzl.la/2Mg8Sli>`__ to convert the account to a + bot and make it the shared review queue for your component +4. BMO administrator updates the email address of the new account to the + ``@mozilla.bugs`` address +5. BMO administrator updates the default reviewer for the component + requested and sets it to the shared review account +6. Reviewers `follow the shared review account in + bugzilla <https://bugzilla.mozilla.org/userprefs.cgi?tab=email>`__ +7. Reviewers get notified when shared review account is ``r?``\ ed |
