blob: e549db9aa3e44dc718e5ffb42fc791629bf21775 (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
|
Metadata-Version: 2.1
Name: taskcluster-taskgraph
Version: 8.0.1
Summary: Build taskcluster taskgraphs
Home-page: https://github.com/taskcluster/taskgraph
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: Console
Classifier: License :: OSI Approved :: Mozilla Public License 2.0 (MPL 2.0)
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development
License-File: LICENSE
Requires-Dist: appdirs >=1.4
Requires-Dist: cookiecutter ~=2.1
Requires-Dist: json-e >=2.7
Requires-Dist: mozilla-repo-urls
Requires-Dist: PyYAML >=5.3.1
Requires-Dist: redo >=2.0
Requires-Dist: requests >=2.25
Requires-Dist: slugid >=2.0
Requires-Dist: taskcluster-urls >=11.0
Requires-Dist: voluptuous >=0.12.1
Provides-Extra: load-image
Requires-Dist: zstandard ; extra == 'load-image'
.. image:: https://firefox-ci-tc.services.mozilla.com/api/github/v1/repository/taskcluster/taskgraph/main/badge.svg
:target: https://firefox-ci-tc.services.mozilla.com/api/github/v1/repository/taskcluster/taskgraph/main/latest
:alt: Task Status
.. image:: https://results.pre-commit.ci/badge/github/taskcluster/taskgraph/main.svg
:target: https://results.pre-commit.ci/latest/github/taskcluster/taskgraph/main
:alt: pre-commit.ci status
.. image:: https://codecov.io/gh/taskcluster/taskgraph/branch/main/graph/badge.svg?token=GJIV52ZQNP
:target: https://codecov.io/gh/taskcluster/taskgraph
:alt: Code Coverage
.. image:: https://badge.fury.io/py/taskcluster-taskgraph.svg
:target: https://badge.fury.io/py/taskcluster-taskgraph
:alt: Pypi Version
.. image:: https://readthedocs.org/projects/taskcluster-taskgraph/badge/?version=latest
:target: https://taskcluster-taskgraph.readthedocs.io/en/latest/?badge=latest
:alt: Documentation Status
.. image:: https://img.shields.io/badge/license-MPL%202.0-orange.svg
:target: http://mozilla.org/MPL/2.0
:alt: License
Taskgraph
=========
Taskgraph is a Python library to generate graphs of tasks for the `Taskcluster
CI`_ service. It is the recommended approach for configuring tasks once your
project outgrows a single `.taskcluster.yml`_ file and is what powers the over
30,000 tasks and counting that make up Firefox's CI.
For more information and usage instructions, `see the docs`_.
How It Works
------------
Taskgraph leverages the fact that Taskcluster is a generic task execution
platform. This means that tasks can be scheduled via its `comprehensive API`_,
and aren't limited to being triggered in response to supported events.
Taskgraph leverages this execution platform to allow CI systems to scale to any
size or complexity.
1. A *decision task* is created via Taskcluster's normal `.taskcluster.yml`_
file. This task invokes ``taskgraph``.
2. Taskgraph evaluates a series of yaml based task definitions (similar to
those other CI offerings provide).
3. Taskgraph applies transforms on top of these task definitions. Transforms
are Python functions that can programmatically alter or even clone a task
definition.
4. Taskgraph applies some optional optimization logic to remove unnecessary
tasks.
5. Taskgraph submits the resulting *task graph* to Taskcluster via its API.
Taskgraph's combination of declarative task configuration combined with
programmatic alteration are what allow it to support CI systems of any scale.
Taskgraph is the library that powers the 30,000+ tasks making up `Firefox's
CI`_.
.. _Taskcluster CI: https://taskcluster.net/
.. _comprehensive API: https://docs.taskcluster.net/docs/reference/platform/queue/api
.. _.taskcluster.yml: https://docs.taskcluster.net/docs/reference/integrations/github/taskcluster-yml-v1
.. _Firefox's CI: https://treeherder.mozilla.org/jobs?repo=mozilla-central
.. _see the docs: https://taskcluster-taskgraph.readthedocs.io
Installation
------------
Taskgraph supports Python 3.8 and up, and can be installed from Pypi:
.. code-block::
pip install taskcluster-taskgraph
Alternatively, the repo can be cloned and installed directly:
.. code-block::
git clone https://github.com/taskcluster/taskgraph
cd taskgraph
python setup.py install
In both cases, it's recommended to use a Python `virtual environment`_.
.. _virtual environment: https://docs.python.org/3/tutorial/venv.html
Get Involved
------------
If you'd like to get involved, please see our `contributing docs`_!
.. _contributing docs: https://github.com/taskcluster/taskgraph/blob/main/CONTRIBUTING.rst
|
