diff options
Diffstat (limited to 'docs/bug-mgmt/policies')
| -rw-r--r-- | docs/bug-mgmt/policies/new-feature-triage.rst | 55 | ||||
| -rw-r--r-- | docs/bug-mgmt/policies/regressions-github.rst | 151 | ||||
| -rw-r--r-- | docs/bug-mgmt/policies/triage-bugzilla.rst | 276 |
3 files changed, 482 insertions, 0 deletions
diff --git a/docs/bug-mgmt/policies/new-feature-triage.rst b/docs/bug-mgmt/policies/new-feature-triage.rst new file mode 100644 index 0000000000..7bc8022777 --- /dev/null +++ b/docs/bug-mgmt/policies/new-feature-triage.rst @@ -0,0 +1,55 @@ +New Feature Triage +================== + +Identifying Feature Requests +---------------------------- + +Bugs which request new features or enhancements should be of +type=\ ``enhancement``. + +Older bugs may also be feature requests if some or all of the following +are true: + +- Bugs with ``feature`` or similar in whiteboard or short description +- ``[RFE]`` in whiteboard, short description, or description +- Bugs not explicitly marked as a feature request, but appear to be + feature requests +- Bugs marked with ``feature`` keyword + +Initial Triage +-------------- + +Staff, contractors, and other contributors looking at new bugs in +*Firefox::Untriaged* and *::General* components should consider if a +bug, if not marked as a feature request, should be one, and if so: + +- Update the bug’s type to ``enhancement`` +- Determine which product and component the bug belongs to and update + it **or** + + - Use *needinfo* to ask a component’s triage owner or a module’s + owner where the request should go + +Product Manager Triage +---------------------- + +- The product manager for the component reviews bugs of type + ``enhancement`` + + - This review should be done a least weekly + +- Reassigns to another Product::Component if necessary **or** +- Determines next steps + + - Close bug as ``RESOLVED WONTFIX`` with comment as to why and + thanking submitter + - If bug is similar enough to work in roadmap, close bug as + ``RESOLVED DUPLICATE`` of the feature bug it is similar to + + - If there’s not a feature bug created already, then consider + making this bug the feature bug + + - Set type to ``enhancement`` + + - Set bug to ``P5`` priority with comment thanking submitter, and + explaining that the request will be considered for future roadmaps diff --git a/docs/bug-mgmt/policies/regressions-github.rst b/docs/bug-mgmt/policies/regressions-github.rst new file mode 100644 index 0000000000..1a3c6b2a4d --- /dev/null +++ b/docs/bug-mgmt/policies/regressions-github.rst @@ -0,0 +1,151 @@ +Regressions from GitHub +======================= + +Release Management and the weekly regression triage must be aware of the +status of all reported regressions in order to assure we are not +shipping known regressions in Firefox releases. + +If a team is using GitHub to manage their part of the Firefox project, +there’s a risk that those groups might not see a regression. + +We need an agreed to standard for how we keep track of these. + +Policy +------ + +*All Firefox components, even if their bugs are tracked off of Bugzilla, +must have a component in Bugzilla.* + +*If a regression bug is found in any of the release trains (Nightly, +Beta, Release, or ESR) and the bug is in a Firefox component which uses +an external repository, the regression must be tracked by a bug in +Bugzilla (whether or not the component in question uses an external +issue tracker).* + +*Unless approved by Release Management, any GitHub managed code with +open regressions will not be merged to mozilla-central. Even if the +regression is not code that has been previously merged into +mozilla-central.* + +*All Firefox code managed in GitHub which uses GitHub to manage +issues* `must use the shared +tags <https://mozilla.github.io/bmo-harmony/labels>`__. + +Comments +~~~~~~~~ + +The bug **must** have the regression keyword. + +The bug **must** have release flags set. + +If the team works in an external bug tracker, then the Bugzilla bug +**must** reference, using the see-also field, the URL of the bug in the +external tracker. + +The bug **must not** be RESOLVED until the code from the external +repository containing the change set for the bug has landed in +mozilla-central. When the change set lands in mozilla-central, the +Bugzilla tracking bug should be set to RESOLVED FIXED and release status +flags should be updated to reflect the release trains the fix has been +landed or uplifted into. + +If the change set containing the patch for the regression is reverted +from mozilla-central, for any reason, then the tracking bug for the +regression **must** be set to REOPENED and the release status flags +updated accordingly. + +If the change set containing the patch for the bug is backed out, for +any reason, the bug must be reopened and the status flags on the +Bugzilla tracking bug updated. + +The team responsible for the component with the regression **should** +strive to create a patch for mozilla-central which contains the fix for +the bug alone, not a monolithic patch containing changes for several +other bugs or features. + +Landings of third-party libraries `must contain a manifest +file <https://docs.google.com/document/d/12ihxPXBo9zBBaU_pBsPrc_wNHds4Upr-PwFfiSHrbu8>`__. + +Best Practices +-------------- + +You must file a regression bug in Bugzilla +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +*If the code with the regression has landed in mozilla-central, you must +file a regression bug.* + +Example +^^^^^^^ + +While using a release of Firefox (Nightly, Beta, Release, ESR) you run +across a bug. Upon research using MozRegression or other tools you find +that the bug was introduced in a change set imported from a component +whose code and issues are managed in GitHub. + +Actions to take +''''''''''''''' + +- Open a new bug in Bugzilla in appropriate component and add the + REGRESSION keyword +- Set affected status for the releases where the bug appears +- Open an issue in the corresponding GitHub project, put the Bugzilla + bug number in the title with the prefix ‘Bug’ (i.e. “Bug 99999: + Regression in foo”) +- Add the REGRESSION label to the new issue +- Add the link to the GitHub issue into the ‘See Also” field in the + Bugzilla bug + +Consequences +'''''''''''' + +*Until the regression is fixed or backed out of the GitHub repo, the +project cannot merge code into mozilla-central* + +Example +^^^^^^^ + +You import a development branch of a component managed in GitHub into +your copy of master. You find a bug and isolate it to the imported +branch. The code is managed in their own GitHub project, but bugs are +managed in Bugzilla. + +Actions to take +''''''''''''''' + +- Open a new bug in Bugzilla in appropriate component and add the + REGRESSION keyword +- Set affected status for the releases where the bug appears + +Consequences +'''''''''''' + +*Until the regression is fixed or backed out of the GitHub repo, the +project cannot merge code into mozilla-central* + +Do not file a regression bug in Bugzilla +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +*If the code with the regression has not landed in mozilla-central, you +do not need to file a bug.* + + +Example +^^^^^^^ + +You import a development branch of a component managed in GitHub into +your copy of master. You find a bug and isolate it to the imported +branch. The code and issues are managed in their own GitHub project. + + +Actions to take +''''''''''''''' + +- File new issue in the GitHub repository of the imported code. +- Label issue as REGRESSION + +Consequence +''''''''''' + +*Issue blocks merge of GitHub project with mozilla-central until +resolved or backed out.* diff --git a/docs/bug-mgmt/policies/triage-bugzilla.rst b/docs/bug-mgmt/policies/triage-bugzilla.rst new file mode 100644 index 0000000000..276979eaf1 --- /dev/null +++ b/docs/bug-mgmt/policies/triage-bugzilla.rst @@ -0,0 +1,276 @@ +Triage for Bugzilla +=================== + +Expectations +------------ + +All teams working on Firefox using either or both Mozilla-central and +Bugzilla are expected to follow the following process. + +What is a Triaged Bug +--------------------- + +The new definition of Triaged will be Firefox-related bugs of type +``defect`` where the component is not +``UNTRIAGED``, and a severity value not equal to ``--`` or ``N/A``. + +Bugs of type Task or Enhancement may have a severity of ``N/A``, +but defects must have a severity that is neither ``--`` or +``N/A``. + +Why Triage +---------- + +We want to make sure that we looked at every defect and a severity has +been defined. This way, we make sure that we did not miss any critical +issues during the development and stabilization cycles. + +Staying on top of the bugs in your component means: + +- You get ahead of critical regressions and crashes which could trigger + a point release if uncaught + + - And you don’t want to spend your holiday with the Release + Management team (not that they don’t like you) + +- Your bug queue is not overwhelming + + - Members of your team do not see the bug queue and get the + ‘wiggins’ + +Who Triages +----------- + +Engineering managers and directors are responsible for naming the +individuals responsible for triaging :ref:`all types of bugs <Bug Types>` in a component. + +We use Bugzilla to manage this. See the `list of triage +owners <https://bugzilla.mozilla.org/page.cgi?id=triage_owners.html>`__. + +If you need to change who is responsible for triaging a bug in a +component, please `file a bug against bugzilla.mozilla.org in the +Administration +component <https://bugzilla.mozilla.org/enter_bug.cgi?product=bugzilla.mozilla.org&component=Administration>`__. +When a new component is created, a triage owner **must** be named. + +Rotating triage +~~~~~~~~~~~~~~~ + +Some components are monitored by a rotation of triagers. In those cases, +the triage owner on Bugzilla will be automatically updated to reflect the +person on the rotation. The rotations are managed as calendars. + +If you wish to set up a rotation for triaging one or more components, +add a link to your rotation calendar in the `triage rotations spreadsheet <https://docs.google.com/spreadsheets/d/1EK6iCtdD8KP4UflIHscuZo6W5er2vy_TX7vsmaaBVd4>`__. + +Firefox::General and Toolkit::General +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Bugs in Firefox::General are fitted with Bug Bug’s model to see if +there’s another component with a high liklihood of fit, and if a +threshold confidence is achieved, the bug is moved to that component. + +Members of the community also review bugs in this component and try to +move them. + +What Do You Triage +------------------ + +As a triage owner the queries you should be following for your component +are: + +- All open bugs, in your components without a pending ``needinfo`` flag + which do not have a valid value of severity set +- All bugs with active review requests in your component which have not + been modified in five days +- All bugs with reviewed, but unlanded patches in your components +- All bugs with a needinfo request unanswered for more than 10 days + +There’s a tool with these queries to help you find bugs +https://mozilla.github.io/triage-center/ and the source is at +https://github.com/mozilla/triage-center/. + +If a bug is an enhancement it needs a priority set and a target release +or program milestone. These bugs are normally reviewed by product +managers. Enhancements can lead to release notes and QA needed that we +also need to know about + +If a bug is a task resulting in a changeset, release managers will need +to known when this work will be done. A task such as refactoring fragile +code can be risky. + +Weekly or More Frequently (depending on the component) find un-triaged +bugs in the components you triage. + +Decide the :ref:`Severity <Defect Severity>` for each untriaged bug +(you can override what’s already been set.) + +These bugs are reviewed in the weekly Regression Triage meeting + +- Bugs of type ``defect`` with the ``regression`` keyword without + ``status-firefoxNN`` decisions +- Bugs of type ``defect`` with the ``regression`` keyword without + a regression range + +Automatic Bug Updates +~~~~~~~~~~~~~~~~~~~~~ + +When a bug is tracked for a release, i.e. the ``tracking_firefoxNN`` +flag is set to ``+`` or ``blocking`` triage decisions will be overridden, +or made as follows: + +- If a bug is tracked for or blocking beta, release or ESR, its + priority will be set to ``P1`` +- If a bug is tracked for or blocking nightly, its priority will be set + to ``P2`` + +Because bugs can be bumped in priority it’s essential that triage owners +review their +`P1 <https://bugzilla.mozilla.org/buglist.cgi?priority=P1&f1=triage_owner&o1=equals&resolution=---&v1=%25user%25>`__ +and +`P2 <https://bugzilla.mozilla.org/buglist.cgi?priority=P2&f1=triage_owner&o1=equals&resolution=---&v1=%25user%25>`__ +bugs frequently. + +Assumptions +~~~~~~~~~~~ + +If a bug's release status in Firefox version N was ``affected`` or ``wontfix``, +its Severity is ``S3`` or ``S4`` and its Priority is ``P3`` or lower (backlog,) +then its release status in Firefox version N+1, if the bug is still open, +is considered to be ``wontfix``. + +Questions and Edge Cases +------------------------ + +This bug is a feature request +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Set the bug’s type to ``enhancement``, add the ``feature`` keyword if +relevant, and state to ``NEW``. Set the bug's Severity to ``N/A``. This +bug will be excluded from future triage queries. + +This bug is a task, not a defect +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Set the bug’s type to ``task``, and state to ``NEW``. Set the bug's +Severity to ``N/A``. This bug will be excluded from future triage queries. + + +If you are not sure of a bug’s type, check :ref:`our rules for bug +types <Bug Types>`. + +This bug’s state is ``UNCONFIRMED`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Are there steps to reproduce? If not, needinfo the person who filed the +bug, requesting steps to reproduce. You are not obligated to wait +forever for a response, and bugs for which open requests for information +go unanswered can be ``RESOLVED`` as ``INCOMPLETE``. + +I need help reproducing the bug +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Set a needinfo for the QA managers, Softvision project managers, or the +QA owner of the component of the bug. + +I don’t have enough information to make a decision +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If you don’t have a reproduction or confirmation, or have questions +about how to proceed, ``needinfo`` the person who filed the bug, or +someone who can answer. + +The ``stalled`` keyword +~~~~~~~~~~~~~~~~~~~~~~~ + +The extreme case of not-enough-information is one which cannot be +answered with a ``needinfo`` request. The reporter has shared all they +know about the bug, we are out of strategies to take to resolve it, but +the bug should be kept open. + +Mark the bug as stalled by adding the ``stalled`` keyword to it. The +keyword will remove it from the list of bugs to be triaged. + +If a patch lands on a ``stalled`` bug, automation will remove the +keyword. Otherwise, when the ``keyword`` is removed, the bug will have +its priority reset to ``--`` and the components triage owner notified by +automation. + +Bugs which remain ``stalled`` for long periods of time should be +reviewed, and closed if necessary. + +Bug is in the wrong Component +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If the bug has a Severity of ``S3``, ``S4``, or ``N/A`` move the what +you think is the correct component, or needinfo the person +responsible for the component to ask them. + +If the bug has a Severity of ``S1`` or ``S2`` then notify Release Management +and contact the triage owner of the component for which you think it belongs to. +We cannot lose track of a high severity bug because it is in the wrong component. + +My project is on GitHub +~~~~~~~~~~~~~~~~~~~~~~~ + +We have :ref:`a guide for GitHub projects to follow <GitHub Metadata Recommendations>` when +triaging. (Note: this guide needs updating.) + +Summary +------- + +Multiple times weekly +~~~~~~~~~~~~~~~~~~~~~ + +Use queries for the components you are responsible for in +https://mozilla.github.io/triage-center/ to find bugs in +need of triage. + +For each untriaged bug: + +- Assign a Severity +- **Do not** assign a ``defect`` a Severity of + ``N/A`` + +You can, but are not required to set the bug's :ref:`Priority <Priority Definitions>`. + +Watch open needinfo flags +~~~~~~~~~~~~~~~~~~~~~~~~~ + +Don’t let open needinfo flags linger for more than two weeks. + +Close minor bugs with unresponded needinfo flags. + +Follow up on needinfo flag requests. + +The `Triage Center tool <https://mozilla.github.io/triage-center/>`__ will help you find these. + +End of Iteration/Release Cycle +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Any open ``S1`` or ``S2`` bugs at the end of the release cycle +will require review by engineering and release management. A +policy on this is forthcoming. + +Optional +^^^^^^^^ + +(The guidelines on bug priority are under review.) + +Are there open P1s? Revisit their priority, +and move to them to the backlog (``P3``) or ``P2``. + +Are there ``P2`` bugs that should move to ``P1`` +for the next cycle? + +Are there ``P2`` you now know are lower priority, +move to ``P3``. + +Are there ``P3`` bugs you now know you won’t get to? +Either demote to ``P5`` (will accept patch) or +resolve as ``WONTFIX``. + +Getting help +------------ + +- Ask in #bug-handling on chat.mozilla.org |