diff options
Diffstat (limited to 'tools/github-sync/readme.md')
| -rw-r--r-- | tools/github-sync/readme.md | 106 |
1 files changed, 0 insertions, 106 deletions
diff --git a/tools/github-sync/readme.md b/tools/github-sync/readme.md deleted file mode 100644 index d691071336..0000000000 --- a/tools/github-sync/readme.md +++ /dev/null @@ -1,106 +0,0 @@ -# Github synchronization scripts - -This tool aims to help synchronizing changes from mozilla-central to Github on pushes. -This is useful for Gecko sub-projects that have Github mirrors, like `gfx/wr` linking to `https://github.com/servo/webrender`. -Originally, the tools were developed in `https://github.com/staktrace/wrupdater`, -then got moved under `gfx/wr/ci-scripts/wrupdater`, -and finally migrated here while also abstracting away from WebRender specifically. - -The main entry point is the `sync-to-github.sh` script that is called with the following arguments: - 1. name of the project, matching the repository under `https://github.com/moz-gfx` user (e.g. `webrender`) - 2. relative folder in mozilla-central, which is the upstream for the changes (e.g. `gfx/wr`) - 3. downstream repository specified as "organization/project-name" (e.g. `servo/webrender`) - 4. name to call for auto-approving the pull request (e.g. `bors` or `@bors-servo`) - -It creates a staging directory at `~/.ghsync` if one doesn't already exist, -and clones the the downstream repo into it. -The script also requires the `GECKO_PATH` environment variable -to point to a mercurial clone of `mozilla-central`, and access to the -taskcluster secrets service to get a Github API token. - -The `sync-to-github.sh` script does some setup steps but the bulk of the actual work -is done by the `converter.py` script. This script scans the mercurial -repository for new changes to the relative folder in m-c, -and adds commits to the git repository corresponding to those changes. -There are some details in the implementation that make it more robust -than simply exporting patches and attempting to reapply them; -in particular it builds a commit tree structure that mirrors what is found in -the `mozilla-central` repository with respect to branches and merges. -So if conflicting changes land on autoland and inbound, and then get -merged, the git repository commits will have the same structure with -a fork/merge in the commit history. This was discovered to be -necessary after a previous version ran into multiple cases where -the simple patch approach didn't really work. - -One of the actions the `converter.py` takes is to find the last sync point -between Github and mozilla-central. This is done based on the following markers: - - commit message containing the string "[ghsync] From https://hg.mozilla.org/mozilla-central/rev/xxx" - - commit message containing the string "[wrupdater] From https://hg.mozilla.org/mozilla-central/rev/xxx" - - commit with tag "mozilla-xxx" -(where xxx is always a mozilla-central hg revision identifier). - -Once the converter is done converting, the `sync-to-github.sh` script -finishes the process by pushing the new commits to the `github-sync` branch -of the `https://github.com/moz-gfx/<project-name>` repository, -and generating a pull request against the downstream repository. It also -leaves a comment on the PR that triggers testing and automatic merge of the PR. -If there is already a pull request (perhaps from a previous run) the -pre-existing PR is force-updated instead. This allows for graceful -handling of scenarios where the PR failed to get merged (e.g. due to -CI failures on the Github side). - -The script is intended to by run by taskcluster for any changes that -touch the relative folder that land on `mozilla-central`. This may mean -that multiple instances of this script run concurrently, or even out -of order (i.e. the task for an older m-c push runs after the task for -a newer m-c push). The script was written with these possibilities in -mind and should be able to eventually recover from any such scenario -automatically (although it may take additional changes to mozilla-central -for such recovery to occur). That being said, the number of pathological -scenarios here is quite large and they were not really tested. - -## Ownership and access - -When this tool is run in Firefox CI, it needs to have push permissions to -the `moz-gfx` github user's account. It gets this permission via a secret token -stored in the Firefox CI taskcluster secrets service. If you need to update -the token, you need to find somebody who is a member of the -[webrender-ci access group](https://people.mozilla.org/a/webrender-ci/). The -Google Drive associated with that access group has additional documentation -on the `moz-gfx` github user and the secret token. - -## Debugging - -To debug the converter.py script, you need to have a hg checkout of -mozilla-central, let's assume it's at $MOZILLA. First create a virtualenv -with the right dependencies installed: - -``` -mkdir -p $HOME/.ghsync -virtualenv --python=python3 $HOME/.ghsync/venv -source $HOME/.ghsync/venv/bin/activate -pip3 install -r $MOZILLA/taskcluster/docker/github-sync/requirements.txt -``` - -Also create a checkout of the downstream github repo and set up a `github-sync` -branch to the point where you want port commits to. For example, for WebRender -you'd do: - -``` -cd $HOME/.ghsync -git clone https://github.com/servo/webrender -cd webrender -git checkout -b github-sync master -``` - -(You can set the github-sync branch to a past revision if you want to replicate -a failure that already got committed). - -Then run the converter from your hg checkout: - -``` -cd $MOZILLA -tools/github-sync/converter.py $HOME/.ghsync/webrender gfx/wr -``` - -You can set the DEBUG variable in the script to True to get more output. |