diff options
Diffstat (limited to 'docs/bug-mgmt/policies')
| -rw-r--r-- | docs/bug-mgmt/policies/attention-dashboard.rst | 108 | ||||
| -rw-r--r-- | docs/bug-mgmt/policies/new-feature-triage.rst | 55 | ||||
| -rw-r--r-- | docs/bug-mgmt/policies/regressions-github.rst | 151 | ||||
| -rw-r--r-- | docs/bug-mgmt/policies/triage-bugzilla.rst | 277 |
4 files changed, 591 insertions, 0 deletions
diff --git a/docs/bug-mgmt/policies/attention-dashboard.rst b/docs/bug-mgmt/policies/attention-dashboard.rst new file mode 100644 index 0000000000..b3f79502e9 --- /dev/null +++ b/docs/bug-mgmt/policies/attention-dashboard.rst @@ -0,0 +1,108 @@ +What Needs My Attention +======================= + +Bugzilla’s `What needs my attention?`_ (a.k.a What Should I Work On Next) dashboard helps us to focus on the top-most important or urgent engineering tasks for releasing Firefox. + +The dashboard is not designed for including everything on a person’s plate. It doesn’t attempt to prioritize normal work - it is just for the things that are more important than the normal stuff. It’s about individual engineer prioritization rather than team prioritization and it doesn’t claim to include all sources of high priority work (e.g. triage, responding to requests from HR, etc). + +The dashboard is available to Mozilla engineers, using the icon + + .. image:: ../assets/icon_assignments.png + :alt: Notification Icon + +in the top right hand corner, after you log into Bugzilla. + +The dashboard is a collection of some (sometimes non-obvious) Bugzilla searches. **Web Platform engineers** should check this page once per day, and ideally keep the list empty, so we can focus on our normal or planned work. + + +.. _What needs my attention?: https://bugzilla.mozilla.org/page.cgi?id=whats_next.html + + +Rules of thumb +-------------- + +Here are some rules of thumb about our priorities when it comes to bug fixing. Specific requests from a manager take precedence over these instructions. + + +Code review requests are not visible on this dashboard; please visit `Phabricator`_ to view those. In general it’s reasonable and important to prioritize Review Requests to unblock others. + + +Follow these general principles and use your best judgment with the help of a manager when necessary. + + +.. _Phabricator: https://phabricator.services.mozilla.com/ + + +Highest priority tasks +~~~~~~~~~~~~~~~~~~~~~~ + +These are the things you should drop everything else for. Generally, work where you block others should be addressed as higher priority than non-blocking work. + +#. Critical needinfos + + * Bugs that are needinfo? you and are marked as Severity = S1 defects or with the “sec-critical” keyword. + + * Bugs that are needinfo? you and are marked as being tracked against or blocking the current beta release. These bugs are potentially S2 once they are triaged and important. If we don’t act on these bugs, we’re in danger of delaying a release. + + * Bugs that are needinfo? you and are marked as a security issue without rating. These bugs are potentially sec-critical or sec-high once they are triaged and important. + + +#. Critical bugs assigned to you + + * Severity = S1 defects and bugs with the “sec-critical” keyword. + + +High priority tasks +~~~~~~~~~~~~~~~~~~~ + +High priority tasks are also “drop everything”, except that in this case “everything” doesn’t include anything in the “Highest priority” list. Generally, work where you block others should be addressed as higher priority than non-blocking work. + +#. Important needinfos + + * Bugs that are needinfo? you and are marked as Severity = S2 defects or with the “sec-high” keyword. + + +#. Important bugs assigned to you + + * Severity = S2 defects and bugs with the “sec-high” keyword (both for things that are not disabled in the current release) + * Note: Some teams have very long lists of S2 defects - see notes below on “Long High Priority task lists” + +#. Your other needinfos (except for things that are self-needinfo). + + +Handling needinfos +------------------ + +TL;DR: Don’t leave people hanging. Reply and convert needinfos from others to self-needinfos. + +Many people have long lists of needinfos. Please don’t ignore them. Here’s how we suggest you burn them down, and keep them down: + +* Any needinfo older than 3 months has probably been forgotten about by the requester. It’s okay to declare needinfo bankruptcy. If you’re concerned about annoying the requester by clearing the needinfo, feel free to point them at this document and offer that they can re-request if there is still a problem. + +* For newer requests, don’t leave someone or something, .e.g BugBot hanging. If you can take action in the short term, do so. If you can’t do it straight away, reply letting the requester know that you intend to work on it, but can’t do so immediately. This clears the original needinfo. At the same time you can re-request a needinfo (using the “Request information from ‘myself’” feature). This converts the needinfo to a self-needinfo, and takes it off the attention list. Be mindful, do the conversion when you have given the right attention and response so that others are not hanging on there, or you've done the best you can for the time being to unblock others. We don't want it to become a way to ignore requests. + + +Review requests +--------------- + +This list doesn’t include Review Requests as as we are still investigating the feasibility of including them and applying these strict rules, but we might consider adding this to a future revision. + +In the meanwhile, it’s worthwhile considering the use of peer review groups set up in Phabricator so that multiple engineers can assist in reviews. + + +Other notes +----------- + +* Long lists of High Priority tasks: For some people and teams, the list of “High Priority” tasks is so long that you would never do normal work. If this is you then you should schedule these tasks alongside normal work. However making your task list manageable should still be a priority. + +* `Severity`_ is defined, but things get a bit hazy when it comes to how we define severity for enhancements; this list is for serious defects only. + +.. _Severity: https://firefox-source-docs.mozilla.org/bug-mgmt/guides/severity.html + + +Everything else +--------------- + +This list is not designed for including everything or prioritizing your normal work. Over time we’d like to bring teams’ practices for prioritizing new work more in line with each other, but that’s not the job of this note. + +If you find that most of your time is spent on high or highest priority tasks, then it’s time to ask some questions to work out why - there’s likely to be a problem behind this and it sounds like a recipe for burnout, and we should do everything we can to even things out. diff --git a/docs/bug-mgmt/policies/new-feature-triage.rst b/docs/bug-mgmt/policies/new-feature-triage.rst new file mode 100644 index 0000000000..7bc8022777 --- /dev/null +++ b/docs/bug-mgmt/policies/new-feature-triage.rst @@ -0,0 +1,55 @@ +New Feature Triage +================== + +Identifying Feature Requests +---------------------------- + +Bugs which request new features or enhancements should be of +type=\ ``enhancement``. + +Older bugs may also be feature requests if some or all of the following +are true: + +- Bugs with ``feature`` or similar in whiteboard or short description +- ``[RFE]`` in whiteboard, short description, or description +- Bugs not explicitly marked as a feature request, but appear to be + feature requests +- Bugs marked with ``feature`` keyword + +Initial Triage +-------------- + +Staff, contractors, and other contributors looking at new bugs in +*Firefox::Untriaged* and *::General* components should consider if a +bug, if not marked as a feature request, should be one, and if so: + +- Update the bug’s type to ``enhancement`` +- Determine which product and component the bug belongs to and update + it **or** + + - Use *needinfo* to ask a component’s triage owner or a module’s + owner where the request should go + +Product Manager Triage +---------------------- + +- The product manager for the component reviews bugs of type + ``enhancement`` + + - This review should be done a least weekly + +- Reassigns to another Product::Component if necessary **or** +- Determines next steps + + - Close bug as ``RESOLVED WONTFIX`` with comment as to why and + thanking submitter + - If bug is similar enough to work in roadmap, close bug as + ``RESOLVED DUPLICATE`` of the feature bug it is similar to + + - If there’s not a feature bug created already, then consider + making this bug the feature bug + + - Set type to ``enhancement`` + + - Set bug to ``P5`` priority with comment thanking submitter, and + explaining that the request will be considered for future roadmaps diff --git a/docs/bug-mgmt/policies/regressions-github.rst b/docs/bug-mgmt/policies/regressions-github.rst new file mode 100644 index 0000000000..a981e79c5e --- /dev/null +++ b/docs/bug-mgmt/policies/regressions-github.rst @@ -0,0 +1,151 @@ +Regressions from GitHub +======================= + +Release Management and the weekly regression triage must be aware of the +status of all reported regressions in order to assure we are not +shipping known regressions in Firefox releases. + +If a team is using GitHub to manage their part of the Firefox project, +there’s a risk that those groups might not see a regression. + +We need an agreed to standard for how we keep track of these. + +Policy +------ + +*All Firefox components, even if their bugs are tracked off of Bugzilla, +must have a component in Bugzilla.* + +*If a regression bug is found in any of the release trains (Nightly, +Beta, Release, or ESR) and the bug is in a Firefox component which uses +an external repository, the regression must be tracked by a bug in +Bugzilla (whether or not the component in question uses an external +issue tracker).* + +*Unless approved by Release Management, any GitHub managed code with +open regressions will not be merged to mozilla-central. Even if the +regression is not code that has been previously merged into +mozilla-central.* + +*All Firefox code managed in GitHub which uses GitHub to manage +issues* `must use the shared +tags <https://mozilla.github.io/bmo-harmony/labels>`__. + +Comments +~~~~~~~~ + +The bug **must** have the regression keyword. + +The bug **must** have release flags set. + +If the team works in an external bug tracker, then the Bugzilla bug +**must** reference, using the see-also field, the URL of the bug in the +external tracker. + +The bug **must not** be RESOLVED until the code from the external +repository containing the change set for the bug has landed in +mozilla-central. When the change set lands in mozilla-central, the +Bugzilla tracking bug should be set to RESOLVED FIXED and release status +flags should be updated to reflect the release trains the fix has been +landed or uplifted into. + +If the change set containing the patch for the regression is reverted +from mozilla-central, for any reason, then the tracking bug for the +regression **must** be set to REOPENED and the release status flags +updated accordingly. + +If the change set containing the patch for the bug is backed out, for +any reason, the bug must be reopened and the status flags on the +Bugzilla tracking bug updated. + +The team responsible for the component with the regression **should** +strive to create a patch for mozilla-central which contains the fix for +the bug alone, not a monolithic patch containing changes for several +other bugs or features. + +Landings of third-party libraries `must contain a manifest +file <https://docs.google.com/document/d/12ihxPXBo9zBBaU_pBsPrc_wNHds4Upr-PwFfiSHrbu8>`__. + +Best Practices +-------------- + +You must file a regression bug in Bugzilla +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +*If the code with the regression has landed in mozilla-central, you must +file a regression bug.* + +Example +^^^^^^^ + +While using a release of Firefox (Nightly, Beta, Release, ESR) you run +across a bug. Upon research using MozRegression or other tools you find +that the bug was introduced in a change set imported from a component +whose code and issues are managed in GitHub. + +Actions to take +''''''''''''''' + +- Open a new bug in Bugzilla in appropriate component and add the + REGRESSION keyword +- Set affected status for the releases where the bug appears +- Open an issue in the corresponding GitHub project, put the Bugzilla + bug number in the title with the prefix ‘Bug’ (i.e. “Bug 99999: + Regression in foo”) +- Add the REGRESSION label to the new issue +- Add the link to the GitHub issue into the ‘See Also” field in the + Bugzilla bug + +Consequences +'''''''''''' + +*Until the regression is fixed or backed out of the GitHub repo, the +project cannot merge code into mozilla-central* + +Example +^^^^^^^ + +You import a development branch of a component managed in GitHub into +your copy of master. You find a bug and isolate it to the imported +branch. The code is managed in their own GitHub project, but bugs are +managed in Bugzilla. + +Actions to take +''''''''''''''' + +- Open a new bug in Bugzilla in appropriate component and add the + REGRESSION keyword +- Set affected status for the releases where the bug appears + +Consequences +'''''''''''' + +*Until the regression is fixed or backed out of the GitHub repo, the +project cannot merge code into mozilla-central* + +Do not file a regression bug in Bugzilla +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +*If the code with the regression has not landed in mozilla-central, you +do not need to file a bug.* + + +Example +^^^^^^^ + +You import a development branch of a component managed in GitHub into +your copy of master. You find a bug and isolate it to the imported +branch. The code and issues are managed in their own GitHub project. + + +Actions to take +''''''''''''''' + +- File new issue in the GitHub repository of the imported code. +- Label issue as REGRESSION + +Consequence +''''''''''' + +*Issue blocks merge of GitHub project with mozilla-central until +resolved or backed out.* diff --git a/docs/bug-mgmt/policies/triage-bugzilla.rst b/docs/bug-mgmt/policies/triage-bugzilla.rst new file mode 100644 index 0000000000..53e26b31ca --- /dev/null +++ b/docs/bug-mgmt/policies/triage-bugzilla.rst @@ -0,0 +1,277 @@ +Triage for Bugzilla +=================== + +Expectations +------------ + +All teams working on Firefox using either or both Mozilla-central and +Bugzilla are expected to follow the following process. + +What is a Triaged Bug +--------------------- + +The new definition of Triaged will be Firefox-related bugs of type +``defect`` where the component is not +``UNTRIAGED``, and a :ref:`Severity <Defect Severity>` value not equal +to ``--`` or ``N/A``. + +Bugs of type Task or Enhancement may have a Severity of ``N/A``, +but defects must have a Severity that is neither ``--`` nor +``N/A``. + +Why Triage +---------- + +We want to make sure that we looked at every defect and a severity has +been defined. This way, we make sure that we did not miss any critical +issues during the development and stabilization cycles. + +Staying on top of the bugs in your component means: + +- You get ahead of critical regressions and crashes which could trigger + a point release if uncaught + + - And you don’t want to spend your holiday with the Release + Management team (not that they don’t like you) + +- Your bug queue is not overwhelming + + - Members of your team do not see the bug queue and get the + ‘wiggins’ + +Who Triages +----------- + +Engineering managers and directors are responsible for naming the +individuals responsible for triaging :ref:`all types of bugs <Bug Types>` in a component. + +We use Bugzilla to manage this. See the `list of triage +owners <https://bugzilla.mozilla.org/page.cgi?id=triage_owners.html>`__. + +If you need to change who is responsible for triaging a bug in a +component, please `file a bug against bugzilla.mozilla.org in the +Administration +component <https://bugzilla.mozilla.org/enter_bug.cgi?product=bugzilla.mozilla.org&component=Administration>`__. +When a new component is created, a triage owner **must** be named. + +Rotating triage +~~~~~~~~~~~~~~~ + +Some components are monitored by a rotation of triagers. In those cases, +the triage owner on Bugzilla will be automatically updated to reflect the +person on the rotation. The rotations are managed as calendars. + +If you wish to set up a rotation for triaging one or more components, +add a link to your rotation calendar in the `triage rotations spreadsheet <https://docs.google.com/spreadsheets/d/1EK6iCtdD8KP4UflIHscuZo6W5er2vy_TX7vsmaaBVd4>`__. + +Firefox::General and Toolkit::General +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Bugs in Firefox::General are fitted with Bug Bug’s model to see if +there’s another component with a high likelihood of fit, and if a +threshold confidence is achieved, the bug is moved to that component. + +Members of the community also review bugs in this component and try to +move them. + +What Do You Triage +------------------ + +As a triage owner the queries you should be following for your component +are: + +- All open bugs, in your components without a pending ``needinfo`` flag + which do not have a valid value of severity set +- All bugs with active review requests in your component which have not + been modified in five days +- All bugs with reviewed, but unlanded patches in your components +- All bugs with a needinfo request unanswered for more than 10 days + +There’s a tool with these queries to help you find bugs +https://bugdash.moz.tools/ and the source is at +https://github.com/mozilla/bugdash/. + +If a bug is an enhancement it needs a priority set and a target release +or program milestone. These bugs are normally reviewed by product +managers. Enhancements can lead to release notes and QA needed that we +also need to know about + +If a bug is a task resulting in a changeset, release managers will need +to known when this work will be done. A task such as refactoring fragile +code can be risky. + +Weekly or More Frequently (depending on the component) find un-triaged +bugs in the components you triage. + +Decide the :ref:`Severity <Defect Severity>` for each untriaged bug +(you can override what’s already been set.) + +These bugs are reviewed in the weekly Regression Triage meeting + +- Bugs of type ``defect`` with the ``regression`` keyword without + ``status-firefoxNN`` decisions +- Bugs of type ``defect`` with the ``regression`` keyword without + a regression range + +Automatic Bug Updates +~~~~~~~~~~~~~~~~~~~~~ + +When a bug is tracked for a release, i.e. the ``tracking_firefoxNN`` +flag is set to ``+`` or ``blocking`` triage decisions will be overridden, +or made as follows: + +- If a bug is tracked for or blocking beta, release or ESR, its + priority will be set to ``P1`` +- If a bug is tracked for or blocking nightly, its priority will be set + to ``P2`` + +Because bugs can be bumped in priority it’s essential that triage owners +review their +`P1 <https://bugzilla.mozilla.org/buglist.cgi?priority=P1&f1=triage_owner&o1=equals&resolution=---&v1=%25user%25>`__ +and +`P2 <https://bugzilla.mozilla.org/buglist.cgi?priority=P2&f1=triage_owner&o1=equals&resolution=---&v1=%25user%25>`__ +bugs frequently. + +Assumptions +~~~~~~~~~~~ + +If a bug's release status in Firefox version N was ``affected`` or ``wontfix``, +its Severity is ``S3`` or ``S4`` and its Priority is ``P3`` or lower (backlog,) +then its release status in Firefox version N+1, if the bug is still open, +is considered to be ``wontfix``. + +Questions and Edge Cases +------------------------ + +This bug is a feature request +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Set the bug’s type to ``enhancement``, add the ``feature`` keyword if +relevant, and state to ``NEW``. Set the bug's Severity to ``N/A``. This +bug will be excluded from future triage queries. + +This bug is a task, not a defect +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Set the bug’s type to ``task``, and state to ``NEW``. Set the bug's +Severity to ``N/A``. This bug will be excluded from future triage queries. + + +If you are not sure of a bug’s type, check :ref:`our rules for bug +types <Bug Types>`. + +This bug’s state is ``UNCONFIRMED`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Are there steps to reproduce? If not, needinfo the person who filed the +bug, requesting steps to reproduce. You are not obligated to wait +forever for a response, and bugs for which open requests for information +go unanswered can be ``RESOLVED`` as ``INCOMPLETE``. + +I need help reproducing the bug +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Set a needinfo for the QA managers, Softvision project managers, or the +QA owner of the component of the bug. + +I don’t have enough information to make a decision +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If you don’t have a reproduction or confirmation, or have questions +about how to proceed, ``needinfo`` the person who filed the bug, or +someone who can answer. + +The ``stalled`` keyword +~~~~~~~~~~~~~~~~~~~~~~~ + +The extreme case of not-enough-information is one which cannot be +answered with a ``needinfo`` request. The reporter has shared all they +know about the bug, we are out of strategies to take to resolve it, but +the bug should be kept open. + +Mark the bug as stalled by adding the ``stalled`` keyword to it. The +keyword will remove it from the list of bugs to be triaged. + +If a patch lands on a ``stalled`` bug, automation will remove the +keyword. Otherwise, when the ``keyword`` is removed, the bug will have +its priority reset to ``--`` and the components triage owner notified by +automation. + +Bugs which remain ``stalled`` for long periods of time should be +reviewed, and closed if necessary. + +Bug is in the wrong Component +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If the bug has a Severity of ``S3``, ``S4``, or ``N/A`` move the what +you think is the correct component, or needinfo the person +responsible for the component to ask them. + +If the bug has a Severity of ``S1`` or ``S2`` then notify Release Management +and contact the triage owner of the component for which you think it belongs to. +We cannot lose track of a high severity bug because it is in the wrong component. + +My project is on GitHub +~~~~~~~~~~~~~~~~~~~~~~~ + +We have :ref:`a guide for GitHub projects to follow <GitHub Metadata Recommendations>` when +triaging. (Note: this guide needs updating.) + +Summary +------- + +Multiple times weekly +~~~~~~~~~~~~~~~~~~~~~ + +Use queries for the components you are responsible for in +https://github.com/mozilla/bugdash/ to find bugs in +need of triage. + +For each untriaged bug: + +- Assign a Severity +- **Do not** assign a ``defect`` a Severity of + ``N/A`` + +You can, but are not required to set the bug's :ref:`Priority <Priority Definitions>`. + +Watch open needinfo flags +~~~~~~~~~~~~~~~~~~~~~~~~~ + +Don’t let open needinfo flags linger for more than two weeks. + +Close minor bugs with unresponded needinfo flags. + +Follow up on needinfo flag requests. + +`BugDash <https://github.com/mozilla/bugdash/>`__ will help you find these. + +End of Iteration/Release Cycle +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Any open ``S1`` or ``S2`` bugs at the end of the release cycle +will require review by engineering and release management. A +policy on this is forthcoming. + +Optional +^^^^^^^^ + +(The guidelines on bug priority are under review.) + +Are there open P1s? Revisit their priority, +and move to them to the backlog (``P3``) or ``P2``. + +Are there ``P2`` bugs that should move to ``P1`` +for the next cycle? + +Are there ``P2`` you now know are lower priority, +move to ``P3``. + +Are there ``P3`` bugs you now know you won’t get to? +Either demote to ``P5`` (will accept patch) or +resolve as ``WONTFIX``. + +Getting help +------------ + +- Ask in #bug-handling on chat.mozilla.org |