horok/irker
1
0
Fork 0
Code Activity

Adding upstream version 2.23+dfsg.

Browse source 
Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>
This commit is contained in:
Daniel Baumann 2025-06-21 08:03:52 +02:00
parent 4274e522c0
commit 1e97beb507
Signed by: daniel.baumann
GPG key ID: BCC918A2ABD66424
21 changed files with 3809 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

12
.gitignore vendored Normal file
View file

@ -0,0 +1,12 @@
# Git clutter
*.orig
# Python bits
/*.pyc
# Man Pages
/*.8
/*.1
# HTML Docs
/*.html

27
COPYING Normal file
View file

@ -0,0 +1,27 @@
BSD LICENSE
Copyright (c) 2015, Eric S. Raymond
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

277
LICENSE Normal file
View file

@ -0,0 +1,277 @@
Eclipse Public License - v 2.0
THE ACCOMPANYING PROGRAM IS PROVIDED UNDER THE TERMS OF THIS ECLIPSE
PUBLIC LICENSE ("AGREEMENT"). ANY USE, REPRODUCTION OR DISTRIBUTION
OF THE PROGRAM CONSTITUTES RECIPIENT'S ACCEPTANCE OF THIS AGREEMENT.
1. DEFINITIONS
"Contribution" means:
a) in the case of the initial Contributor, the initial content
Distributed under this Agreement, and
b) in the case of each subsequent Contributor:
i) changes to the Program, and
ii) additions to the Program;
where such changes and/or additions to the Program originate from
and are Distributed by that particular Contributor. A Contribution
"originates" from a Contributor if it was added to the Program by
such Contributor itself or anyone acting on such Contributor's behalf.
Contributions do not include changes or additions to the Program that
are not Modified Works.
"Contributor" means any person or entity that Distributes the Program.
"Licensed Patents" mean patent claims licensable by a Contributor which
are necessarily infringed by the use or sale of its Contribution alone
or when combined with the Program.
"Program" means the Contributions Distributed in accordance with this
Agreement.
"Recipient" means anyone who receives the Program under this Agreement
or any Secondary License (as applicable), including Contributors.
"Derivative Works" shall mean any work, whether in Source Code or other
form, that is based on (or derived from) the Program and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship.
"Modified Works" shall mean any work in Source Code or other form that
results from an addition to, deletion from, or modification of the
contents of the Program, including, for purposes of clarity any new file
in Source Code form that contains any contents of the Program. Modified
Works shall not include works that contain only declarations,
interfaces, types, classes, structures, or files of the Program solely
in each case in order to link to, bind by name, or subclass the Program
or Modified Works thereof.
"Distribute" means the acts of a) distributing or b) making available
in any manner that enables the transfer of a copy.
"Source Code" means the form of a Program preferred for making
modifications, including but not limited to software source code,
documentation source, and configuration files.
"Secondary License" means either the GNU General Public License,
Version 2.0, or any later versions of that license, including any
exceptions or additional permissions as identified by the initial
Contributor.
2. GRANT OF RIGHTS
a) Subject to the terms of this Agreement, each Contributor hereby
grants Recipient a non-exclusive, worldwide, royalty-free copyright
license to reproduce, prepare Derivative Works of, publicly display,
publicly perform, Distribute and sublicense the Contribution of such
Contributor, if any, and such Derivative Works.
b) Subject to the terms of this Agreement, each Contributor hereby
grants Recipient a non-exclusive, worldwide, royalty-free patent
license under Licensed Patents to make, use, sell, offer to sell,
import and otherwise transfer the Contribution of such Contributor,
if any, in Source Code or other form. This patent license shall
apply to the combination of the Contribution and the Program if, at
the time the Contribution is added by the Contributor, such addition
of the Contribution causes such combination to be covered by the
Licensed Patents. The patent license shall not apply to any other
combinations which include the Contribution. No hardware per se is
licensed hereunder.
c) Recipient understands that although each Contributor grants the
licenses to its Contributions set forth herein, no assurances are
provided by any Contributor that the Program does not infringe the
patent or other intellectual property rights of any other entity.
Each Contributor disclaims any liability to Recipient for claims
brought by any other entity based on infringement of intellectual
property rights or otherwise. As a condition to exercising the
rights and licenses granted hereunder, each Recipient hereby
assumes sole responsibility to secure any other intellectual
property rights needed, if any. For example, if a third party
patent license is required to allow Recipient to Distribute the
Program, it is Recipient's responsibility to acquire that license
before distributing the Program.
d) Each Contributor represents that to its knowledge it has
sufficient copyright rights in its Contribution, if any, to grant
the copyright license set forth in this Agreement.
e) Notwithstanding the terms of any Secondary License, no
Contributor makes additional grants to any Recipient (other than
those set forth in this Agreement) as a result of such Recipient's
receipt of the Program under the terms of a Secondary License
(if permitted under the terms of Section 3).
3. REQUIREMENTS
3.1 If a Contributor Distributes the Program in any form, then:
a) the Program must also be made available as Source Code, in
accordance with section 3.2, and the Contributor must accompany
the Program with a statement that the Source Code for the Program
is available under this Agreement, and informs Recipients how to
obtain it in a reasonable manner on or through a medium customarily
used for software exchange; and
b) the Contributor may Distribute the Program under a license
different than this Agreement, provided that such license:
i) effectively disclaims on behalf of all other Contributors all
warranties and conditions, express and implied, including
warranties or conditions of title and non-infringement, and
implied warranties or conditions of merchantability and fitness
for a particular purpose;
ii) effectively excludes on behalf of all other Contributors all
liability for damages, including direct, indirect, special,
incidental and consequential damages, such as lost profits;
iii) does not attempt to limit or alter the recipients' rights
in the Source Code under section 3.2; and
iv) requires any subsequent distribution of the Program by any
party to be under a license that satisfies the requirements
of this section 3.
3.2 When the Program is Distributed as Source Code:
a) it must be made available under this Agreement, or if the
Program (i) is combined with other material in a separate file or
files made available under a Secondary License, and (ii) the initial
Contributor attached to the Source Code the notice described in
Exhibit A of this Agreement, then the Program may be made available
under the terms of such Secondary Licenses, and
b) a copy of this Agreement must be included with each copy of
the Program.
3.3 Contributors may not remove or alter any copyright, patent,
trademark, attribution notices, disclaimers of warranty, or limitations
of liability ("notices") contained within the Program from any copy of
the Program which they Distribute, provided that Contributors may add
their own appropriate notices.
4. COMMERCIAL DISTRIBUTION
Commercial distributors of software may accept certain responsibilities
with respect to end users, business partners and the like. While this
license is intended to facilitate the commercial use of the Program,
the Contributor who includes the Program in a commercial product
offering should do so in a manner which does not create potential
liability for other Contributors. Therefore, if a Contributor includes
the Program in a commercial product offering, such Contributor
("Commercial Contributor") hereby agrees to defend and indemnify every
other Contributor ("Indemnified Contributor") against any losses,
damages and costs (collectively "Losses") arising from claims, lawsuits
and other legal actions brought by a third party against the Indemnified
Contributor to the extent caused by the acts or omissions of such
Commercial Contributor in connection with its distribution of the Program
in a commercial product offering. The obligations in this section do not
apply to any claims or Losses relating to any actual or alleged
intellectual property infringement. In order to qualify, an Indemnified
Contributor must: a) promptly notify the Commercial Contributor in
writing of such claim, and b) allow the Commercial Contributor to control,
and cooperate with the Commercial Contributor in, the defense and any
related settlement negotiations. The Indemnified Contributor may
participate in any such claim at its own expense.
For example, a Contributor might include the Program in a commercial
product offering, Product X. That Contributor is then a Commercial
Contributor. If that Commercial Contributor then makes performance
claims, or offers warranties related to Product X, those performance
claims and warranties are such Commercial Contributor's responsibility
alone. Under this section, the Commercial Contributor would have to
defend claims against the other Contributors related to those performance
claims and warranties, and if a court requires any other Contributor to
pay any damages as a result, the Commercial Contributor must pay
those damages.
5. NO WARRANTY
EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, AND TO THE EXTENT
PERMITTED BY APPLICABLE LAW, THE PROGRAM IS PROVIDED ON AN "AS IS"
BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, EITHER EXPRESS OR
IMPLIED INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OR CONDITIONS OF
TITLE, NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR
PURPOSE. Each Recipient is solely responsible for determining the
appropriateness of using and distributing the Program and assumes all
risks associated with its exercise of rights under this Agreement,
including but not limited to the risks and costs of program errors,
compliance with applicable laws, damage to or loss of data, programs
or equipment, and unavailability or interruption of operations.
6. DISCLAIMER OF LIABILITY
EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, AND TO THE EXTENT
PERMITTED BY APPLICABLE LAW, NEITHER RECIPIENT NOR ANY CONTRIBUTORS
SHALL HAVE ANY LIABILITY FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING WITHOUT LIMITATION LOST
PROFITS), HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OR DISTRIBUTION OF THE PROGRAM OR THE
EXERCISE OF ANY RIGHTS GRANTED HEREUNDER, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES.
7. GENERAL
If any provision of this Agreement is invalid or unenforceable under
applicable law, it shall not affect the validity or enforceability of
the remainder of the terms of this Agreement, and without further
action by the parties hereto, such provision shall be reformed to the
minimum extent necessary to make such provision valid and enforceable.
If Recipient institutes patent litigation against any entity
(including a cross-claim or counterclaim in a lawsuit) alleging that the
Program itself (excluding combinations of the Program with other software
or hardware) infringes such Recipient's patent(s), then such Recipient's
rights granted under Section 2(b) shall terminate as of the date such
litigation is filed.
All Recipient's rights under this Agreement shall terminate if it
fails to comply with any of the material terms or conditions of this
Agreement and does not cure such failure in a reasonable period of
time after becoming aware of such noncompliance. If all Recipient's
rights under this Agreement terminate, Recipient agrees to cease use
and distribution of the Program as soon as reasonably practicable.
However, Recipient's obligations under this Agreement and any licenses
granted by Recipient relating to the Program shall continue and survive.
Everyone is permitted to copy and distribute copies of this Agreement,
but in order to avoid inconsistency the Agreement is copyrighted and
may only be modified in the following manner. The Agreement Steward
reserves the right to publish new versions (including revisions) of
this Agreement from time to time. No one other than the Agreement
Steward has the right to modify this Agreement. The Eclipse Foundation
is the initial Agreement Steward. The Eclipse Foundation may assign the
responsibility to serve as the Agreement Steward to a suitable separate
entity. Each new version of the Agreement will be given a distinguishing
version number. The Program (including Contributions) may always be
Distributed subject to the version of the Agreement under which it was
received. In addition, after a new version of the Agreement is published,
Contributor may elect to Distribute the Program (including its
Contributions) under the new version.
Except as expressly stated in Sections 2(a) and 2(b) above, Recipient
receives no rights or licenses to the intellectual property of any
Contributor under this Agreement, whether expressly, by implication,
estoppel or otherwise. All rights in the Program not expressly granted
under this Agreement are reserved. Nothing in this Agreement is intended
to be enforceable by any entity that is not a Contributor or Recipient.
No third-party beneficiary rights are created under this Agreement.
Exhibit A - Form of Secondary Licenses Notice
"This Source Code may also be made available under the following
Secondary Licenses when the conditions for such availability set forth
in the Eclipse Public License, v. 2.0 are satisfied: {name license(s),
version(s), and exceptions or additional permissions here}."
Simply including a copy of this Agreement, including this Exhibit A
is not sufficient to license the Source Code under Secondary Licenses.
If it is not possible or desirable to put the notice in a particular
file, then You may include the notice in a location (such as a LICENSE
file in a relevant directory) where a recipient would be likely to
look for such a notice.
You may add additional accurate notices of copyright ownership.

113
Makefile Normal file
View file

@ -0,0 +1,113 @@
# Makefile for the irker relaying daemon
VERS := $(shell sed -n 's/version = "\(.\+\)"/\1/p' irkerd)
SYSTEMDSYSTEMUNITDIR := $(shell pkg-config --variable=systemdsystemunitdir systemd)
# `prefix`, `mandir` & `DESTDIR` can and should be set on the command
# line to control installation locations
prefix ?= /usr
mandir ?= /share/man
target = $(DESTDIR)$(prefix)
docs: irkerd.html irkerd.8 irkerhook.html irkerhook.1 irk.html irk.1
irkerd.8: irkerd.xml
xmlto man irkerd.xml
irkerd.html: irkerd.xml
xmlto html-nochunks irkerd.xml
irkerhook.1: irkerhook.xml
xmlto man irkerhook.xml
irkerhook.html: irkerhook.xml
xmlto html-nochunks irkerhook.xml
irk.1: irk.xml
xmlto man irk.xml
irk.html: irk.xml
xmlto html-nochunks irk.xml
install.html: install.adoc
asciidoc -o install.html install.adoc
security.html: security.adoc
asciidoc -o security.html security.adoc
hacking.html: hacking.adoc
asciidoc -o hacking.html hacking.adoc
install: irk.1 irkerd.8 irkerhook.1 uninstall
install -m 755 -o 0 -g 0 -d "$(target)/bin"
install -m 755 -o 0 -g 0 irkerd "$(target)/bin/irkerd"
ifneq ($(strip $(SYSTEMDSYSTEMUNITDIR)),)
install -m 755 -o 0 -g 0 -d "$(DESTDIR)$(SYSTEMDSYSTEMUNITDIR)"
install -m 644 -o 0 -g 0 irkerd.service "$(DESTDIR)$(SYSTEMDSYSTEMUNITDIR)"
endif
install -m 755 -o 0 -g 0 -d "$(target)$(mandir)/man8"
install -m 755 -o 0 -g 0 irkerd.8 "$(target)$(mandir)/man8/irkerd.8"
install -m 755 -o 0 -g 0 -d "$(target)$(mandir)/man1"
install -m 755 -o 0 -g 0 irkerhook.1 "$(target)$(mandir)/man1/irkerhook.1"
install -m 755 -o 0 -g 0 irk.1 "$(target)$(mandir)/man1/irk.1"
uninstall:
rm -f "$(target)/bin/irkerd"
ifneq ($(strip $(SYSTEMDSYSTEMUNITDIR)),)
rm -f "$(DESTDIR)$(SYSTEMDSYSTEMUNITDIR)/irkerd.service"
endif
rm -f "$(target)$(mandir)/man8/irkerd.8"
rm -f "$(target)$(mandir)/man1/irkerhook.1"
rm -f "$(target)$(mandir)/man1/irk.1"
clean:
rm -f irkerd.8 irkerhook.1 irk.1 irker-*.tar.gz *~ *.html
pylint:
@pylint --score=n irkerd irkerhook.py
loc:
@echo "LOC:"; wc -l irkerd irkerhook.py
@echo -n "LLOC: "; grep -vE '(^ *#|^ *$$)' irkerd irkerhook.py | wc -l
DOCS = \
README \
COPYING \
NEWS \
install.adoc \
security.adoc \
hacking.adoc \
irkerhook.xml \
irkerd.xml \
irk.xml \
SOURCES = \
$(DOCS) \
irkerd \
irkerhook.py \
filter-example.py \
filter-test.py \
irk \
Makefile
EXTRA_DIST = \
org.catb.irkerd.plist \
irkerd.service \
irker-logo.png
version:
@echo $(VERS)
irker-$(VERS).tar.gz: $(SOURCES) irkerd.8 irkerhook.1 irk.1
mkdir irker-$(VERS)
cp -pR $(SOURCES) $(EXTRA_DIST) irker-$(VERS)/
@COPYFILE_DISABLE=1 tar -cvzf irker-$(VERS).tar.gz irker-$(VERS)
rm -fr irker-$(VERS)
irker-$(VERS).md5:
@md5sum irker-$(VERS).tar.gz >irker-$(VERS).md5
dist: irker-$(VERS).tar.gz irker-$(VERS).md5
WEBDOCS = irkerd.html irk.html irkerhook.html install.html security.html hacking.html
release: irker-$(VERS).tar.gz irker-$(VERS).md5 $(WEBDOCS)
shipper version=$(VERS) | sh -e -x
refresh: $(WEBDOCS)
shipper -N -w version=$(VERS) | sh -e -x

195
NEWS Normal file
View file

@ -0,0 +1,195 @@
irker history
2.23: 2023-01-27::
Fix typo in support for IPv6 listening.
2.22: 2022-03-15::
Add support for IPv6 listening.
2.21: 2022-01-25::
Restore function of immediate option.
2.20: 2021-09-20
Added --posord-file option
Add socket connection-timeout option.
Ubuntu deleted /usr/bin/python, change all invocations to Python 3.
2.19: 2020-06-29
Codebase is now fully forward-poerted to Python 3.
2.18: 2016-06-02
Add the ability to set the notification-message template (Debian bug #824512)
2.17: 2016-03-14
Add a reconnect delay (Debian bug #749650).
Add proxy support (requres setting some variables in the source file).
Use git abbreviated hash to address Debian complaints.
2.16: 2016-02-18
Code now runs under either Python 2 or Python 3
2.15: 2016-01-12
Emergency backout of getaddrinfo, it randomly hangs.
2.14: 2016-01-12
Lookup with getaddrinfo allows use with IPv6.
Documentation improvements.
2.13: 2015-06-14
SSL validation fix.
Hardening against Unicode decode errors.
irk becomes a library so it can be re-used.
2.12: 2014-10-22
Catch erroneous UTF-8 or non-UTF-8 from servers.
Also autodetect the right logging device under FreeBSD: /var/run/syslog
2.11: 2014-06-20
With -i, message string argument now optional, stdin is read if it is absent.
Auto-adapt to BSD & OS X log device as well as Linux's.
2.10: 2014-06-19
irk no longer fails on ircs channel URLs.
2.9: 2014-06-01
If irkerd is running in background, log to /dev/syslog (facility daemon).
New -H option to set host listening address.
Add support for using CertFP to auth to the IRC server, and document it.
2.8: 2014-05-30
Various minor improvements to irk.
Cope better with branch names containing slashes.
2.7: 2014-03-15
Add support for ircs:// and SSL/TLS connections to IRC servers.
Add support for per-URL usernames and passwords.
2.6: 2014-02-04
Fix for an infinite loop on failing to connect to IRC
2.5: 2013-12-24
Bug fix - remove a deadlock we inherited from irclib.
2.4: 2013-12-03
Bug fix release - some users reported failure to connect with 2.3.
Also prevent a crash if Unicode shows up in the wrong place.
2.3: 2013-11-30
-i option enables immediate sending of one line in foreground.
2.2: 2013-11-29
Fixed Unicode processing - got busted in 2.0 when irclib was removed.
Show Python traceback on higher debug levels.
2.1: 2013-11-26
A performance improvement in the git repository hook.
Documentation polishing.
2.0: 2013-11-16
The dependency on irclib is gone.
An email delivery method, suitable for use on SourceForge.
irkerhook can now be used as a hg changegroup hook.
Prevent misbehavior on UTF-8 in commit metadata.
Fix a crash bug on invalid hostnames.
1.20: 2013-05-17
Compatibility back to Python 2.4 (provided simplejson is present).
Increased anti-flood delay to avoid trouble with freenode.
1.19: 2013-05-06
Fixed a minor bug in argument processing
1.18: 2013-04-16
Added -l option; irker can now be used as a channel monitor.
Added -n and -p option: the nick can be forced and authenticated.
1.17: 2013-02-03
Various minor fixes and bulletproofing.
1.16: 2013-01-24
Deal gracefully with non-ASCII author names and '|' in the command line.
1.15: 2012-12-08
Don't append an extra newline in the Subversion hook.
1.14: 2012-11-26
irclib 5.0 and urlparse compatibility fixes.
1.13: 2012-11-06
Fix for a very rare thread race found by AI0867.
Work around a midesign in the IRC library.
1.12: 2012-10-11
Emergency workaround for a Unicode-handling error buried deep in irclib.
The IRC library at version 3.2 or later is required for this version!
Only ship to freenode #commits by default.
1.11: 2012-10-10
Code is now fully Unicode-safe.
A 'cialike' option emulates the file-summary behavior on the old CIA service.
1.10: 2012-10-09
Expire disconnected connections if they aren't needed or can't reconnect.
Eventlet support removed - didn't play well with the library mutex.
1.9: 2012-10-08
Proper mutex locks prevent an occasional thread crash on session timeout.
There's now systemd installation support for irkerd.
1.8: 2012-10-06
It's now possible to send to nick URLs.
Cope gracefully if an IRC server dies or hangs during the nick handshake.
1.7: 2012-10-05
Optional metadata filtering with a user-specified command.
irkerd code is now armored against IRC library errors in the delivery threads.
1.6: 2012-10-04
In 1.5 trying to appease pylint broke the Mercurial hook.
Added credits for contributors in hacking.txt.
Fix the aging out of connections when we hit a resource limit.
1.5: 2012-10-03
Mercurial support.
Shorten nick negotiation by choosing a random nick base from a large range.
Make irkerd exit cleanly on control-C.
1.4: 2012-10-02
Graceful handling of server disconnects and kicks.
Distribution now inclues an installable irkerd plist for Mac OS/X.
The color variable is no longer boolean; may be miRC or ANSI.
The installation instructions for irkerhook.py have changed!
1.3: 2012-10-01
Support for an irker.conf file to set irkerhook variables under Subversion.
Color highlighting of notification fields can be enabled.
irkerhook.py now has its own manual page.
Added channelmax variable for rate-limiting.
irkerd now uses green threads, with much lower overhead.
Fix a bug in handling of channel names with no prefix.
1.2: 2012-09-30
All segments of a message with embedded newlines are now transmitted.
Message reduction - irkerhook drops the filelist on excessively long ones.
Shell quote hardening in irkerhook.py and some anti-DoS logic.
1.1: 2012-09-28
Add a delay to avoid threads spinning on the empty-queue-check, eating CPU.
Fix a bug in reporting of multi-file commits.
1.0: 2012-09-27
First production version, somewhat rushed by the sudden death of cia.vc
on 24 September.

24
README Normal file
View file

@ -0,0 +1,24 @@
irker - submission tools for IRC notifications
irkerd is a specialized IRC client that runs as a daemon, allowing
other programs to ship IRC notifications by sending JSON objects to a
listening socket.
It is meant to be used by hook scripts in version-control
repositories, allowing them to send commit notifications to project
IRC channels. A hook script, irkerhook.py, supporting git, hg, and
Subversion is included in the distribution; see the install.adoc file
for installation instructions.
The advantage of using this daemon over individual scripted sends
is that it can maintain connection state for multiple channels,
avoiding obnoxious join/leave spam.
The file install.adoc describes how to install the software safely, so
it can't be used as a spam conduit.
Please read the files security.adoc and hacking.adoc before modifying
this code.
Eric S. Raymond
September 2012

29
control Normal file
View file

@ -0,0 +1,29 @@
# This is not a real Debian control file, though the syntax is compatible.
# It's project metadata for the shipper tool
Package: irker
Description: An IRC client that runs as a daemon accepting notification requests.
You present them JSON objects presented to a listening socket. It is
meant to be used by hook scripts in version-control repositories,
allowing them to send commit notifications to project IRC channels.
A hook script that works with git, hg, and svn is included in the
distribution.
XBS-Destinations: mailto:patrick@gentoo.org
Homepage: mailto:packages@qa.debian.org
XBS-HTML-Target: index.html
XBS-Repository-URL: https://gitlab.com/esr/irker
XBS-Debian-Packages: irker
XBS-OpenHub-URL: http://www.openhub.net/p/irker
XBS-IRC-Channel: irc://chat.freenode.net/#irker
XBS-Logo: irker-logo.png
XBS-VC-Tag-Template: %(version)s

13
filter-example.py Executable file
View file

@ -0,0 +1,13 @@
#!/usr/bin/env python3
# This is a trivial example of a metadata filter.
# All it does is change the name of the commit's author.
# It could do other things, including modifying the
# channels list
#
import sys, json
metadata = json.loads(sys.argv[1])
metadata['author'] = "The Great and Powerful Oz"
print json.dumps(metadata)
# end

35
filter-test.py Executable file
View file

@ -0,0 +1,35 @@
#!/usr/bin/env python3
#
# Test hook to launch an irker instance (if it doesn't already exist)
# just before shipping the notification. We start it in in another terminal
# so you can watch the debug messages. Intended to be used in the root
# directory of the irker repo. Probably only of interest only to irker
# developers
#
# To use this, set up irkerhook.py to fire on each commit. Creating a
# .git/hooks/post-commit file containing the line "irkerhook.py"; be
# sure to make the opos-commit file executable. Then set the
# filtercmd variable in your repo config as follows:
#
# [irker]
# filtercmd = filter-test.py
import os, sys, json, subprocess, time
metadata = json.loads(sys.argv[1])
ps = subprocess.Popen("ps -U %s uh" % os.getenv("LOGNAME"),
shell=True,
stdout=subprocess.PIPE)
data = ps.stdout.read()
irkerd_count = len([x for x in data.split("\n") if x.find("irkerd") != -1])
if irkerd_count:
sys.stderr.write("Using a running irker instance...\n")
else:
sys.stderr.write("Launching a new irker instance...\n")
os.system("gnome-terminal --title 'irkerd' -e 'irkerd -d 2' &")
time.sleep(1.5) # Avoid a race condition
print json.dumps(metadata)
# end

80
hacking.adoc Normal file
View file

@ -0,0 +1,80 @@
= Hacker's Guide to irker =
== Design philosopy ==
Points to you if some of this seems familiar from GPSD...
=== Keep mechanism and policy separate ===
Mechanism goes in irkerd. Policy goes in irkerhook.py
irkerd is intended to be super-simple and completely indifferent to
what content passes through it. It doesn't know, in any sense, that
the use-case it was designed for is broadcasting notifications from
version control systems. irkerhook.py is the part that knows about how
to mine data from repositories and sets the format of notifications.
=== If you think the mechanism needs an option, think again ===
Because irkerhook.py does policy, it takes policy options. Because
irkerd is pure mechanism, it shouldn't need any. If you think it
does, you have almost certainly got a bug in your thinking. Fix
that before you modify code.
=== Never configure what you can autoconfigure ===
Human attention is more expensive than machine time. Humans are
careless and failure-prone. Therefore, whenever you make a user tell
your code something the code can deduce for itself, you are
introducing unnecessary inefficiency and unnecessary failure modes.
This, in particular, is why irkerhook.py doesn't have a repository
type switch. It can deduce the repo type by looking, so it should.
== Release procedure ==
1. Check for merge requests at the repository.
2. Do 'make pylint' to audit the code.
3. Run irk with a sample message; look at #irker on freenode to verify.
4. Bump the version numbers in irkerd and irkerhook.py
5. Run irkerhook.py -n and verify that the output looks sane.
6. Update the NEWS file
7. git commit -a
8. make release
== Thanks where due ==
Alexander van Gessel (AI0867) <ai0867@gmail.com> contributed the
Subversion support in irkerhook.py. Since the 1.0 release he has
kept as close an eye on the code as the author and has fixed at least
as many bugs.
//W. here causes asciidoc to see this as a list entry.
W Trevor King <wking@tremily.us> added SSL/TLS support and did
significant refactoring work.
Daniel Franke <dfoxfranke@gmail.com> performed a security audit of irkerd.
Georg Brandl <georg@python.org> contributed the Mercurial support in
irkerhook.py and explained how to make Control-C work right.
Laurent Bachelier <laurent@bachelier.name> fixed the Makefile so it
wouldn't break stuff and wrote the first version of the external
filtering option.
dak180 (name withheld by request) wrote the OS X launchd plist.
Wulf C. Krueger <philantrop@exherbo.org> wrote the systemd
installation support.
Other people on the freenode #irker channel (Kingpin, fpcfan,
shadowm, Rick) smoked out bugs in irkerd before they could seriously
bug anybody.

116
install.adoc Normal file
View file

@ -0,0 +1,116 @@
= Forge installation instructions =
irker and irkerhook.py are intended to be installed on forge sites
such as SourceForge, GitHub, GitLab, Gna, and Savannah. This
file explains the theory of operation, how to install the code,
and how to test it.
== Prerequisites ==
You should have Python 3 installed. While Python 2 support
has not yet been removed, it is unmaintained and vulnerable
to bitrot.
If you just want to use irkerd and/or irkerhook.py,
you need not bother with the Makefile. It's for building
the derived versions of the documebtation and rubnning
validation tools.
If you want to run irkerd using a socket proxy,
you'll want to do this:
-------------------------------------
pip install -r requirements.txt
-------------------------------------
Otherwise the code has no dependencies outside
the Python standard library.
== Theory of operation ==
irkerhook.py creates JSON notification requests and ships them to
irkerd's listener socket. irkerd run as a daemon in order to maintain
all the client state required to post multiple notifications while generating
a minimum of join/leave messages (which, from the point of view of
humans watching irkerd's output, are mere spam).
See the security.txt document for a detailed discussion of security
and DoS vulnerabilities related to irker. The short version: as
long as your firewall blocks port 6659 and irkerd is running inside
it, you should be fine.
== Prerequisites ==
You will need either
1. Python at version 2.6 or later, which has JSON built in
2. Python at version no older than 2.4, and a version of the
simplejson library installed that it can use. Some newer
versions of simplejson discard 2.4 compatibility; 2.0.9
is known to work.
== Installing irkerd ==
irker needs to run constantly, watching for TCP and UDP traffic on
port 6659. Install it accordingly. It has no config file; you can
just start it up with no arguments. If you want to see what it's
doing, give it command-line options -d info for sparse messages and
-d debug to show all traffic with IRC servers.
You should *not* make irker visible from outside the site firewall, as
it can be used to spam IRC channels while masking the source address.
The firewall should block port 6659.
The design of irker assumes the machine on which it is running is also
inside the firewall, so that repository hooks can reach port 6659.
The file org.catb.irkerd.plist is a Mac OS/X plist that can be
installed to launch irkerd as a boot-time service on that system.
irker.service is a systemd unit that can run irkerd as a boot-time
service on systems that support systemd. This is configured to
run irkerd under a separate user account (irker), so this needs to
be set up before starting irker, or the unit needs to be modified
to use a different user.
== Installing irkerhook.py ==
Under git, a call to irkerhook.py should be installed in the update
hook script of your repo. Under Subversion, the call goes in your
repo's post-commit script. Under Mercurial there are two different
ways to install it. See the irkerhook manual page for details; the
source is irkerhook.xml in this distribution.
SourceForge is a special case: see
https://github.com/AI0867/sf-git-irker-pipeline
for tools and instructions on how to work around its limitations.
== Testing ==
To verify that your repo produces well-formed JSON notifications,
you can run irkerhook.py in the repo directory using the -n switch,
which emits JSON to standard output rather than attempting to ship
to an irkerd instance.
Then, start irkerd and call irkerhook.py while watching the freenode
#commits channel.
The 'irk' script is a little test tool that takes two arguments,
a channel and a message, and does what you'd expect.
If you need help, there's a project chat channel at
irc://chat.freenode.net/#irker
== Read-only access ==
If, for whatever reason, you can't modify the hook scripts in your
repository, there is still hope. There's a poller daemon that can
watch activity in a Subversion repository and ship notifications via
an irker instance.
https://github.com/shikadilord/irker-svnpoller

65
irk Executable file
View file

@ -0,0 +1,65 @@
#!/usr/bin/env python3
# Illustrates how to test irkerd.
#
# First argument must be a channel URL. If it does not begin with "irc",
# the base URL for freenode is prepended.
#
# Second argument must be a payload string. Standard C-style escapes
# such as \n and \t are decoded.
#
# SPDX-License-Identifier: BSD-2-Clause
import json
import socket
import sys
import fileinput
DEFAULT_SERVER = ("localhost", 6659)
def connect(server = DEFAULT_SERVER):
return socket.create_connection(server)
def send(s, target, message):
data = {"to": target, "privmsg" : message}
dump = json.dumps(data)
if not isinstance(dump, bytes):
dump = dump.encode('ascii')
s.sendall(dump)
def irk(target, message, server = DEFAULT_SERVER):
s = connect(server)
if "irc:" not in target and "ircs:" not in target:
target = "irc://chat.freenode.net/{0}".format(target)
if message == '-':
for line in fileinput.input('-'):
send(s, target, line.rstrip('\n'))
else:
send(s, target, message)
s.close()
def main():
if len(sys.argv) < 2:
sys.stderr.write("irk: a URL argument is required\n")
sys.exit(1)
target = sys.argv[1]
message = " ".join(sys.argv[2:])
# Allows pretty formatting of irker messages
if str == bytes:
message = message.decode('string_escape')
# The actual IRC limit is 512. Avoid any off-by-ones
chunksize = 511
try:
while message[:chunksize]:
irk(target, message[:chunksize])
message = message[chunksize:]
except socket.error as e:
sys.stderr.write("irk: write to server failed: %r\n" % e)
sys.exit(1)
if __name__ == '__main__':
main()
# The following sets edit modes for GNU EMACS
# Local Variables:
# mode:python
# End:

84
irk.xml Normal file
View file

@ -0,0 +1,84 @@
<!DOCTYPE refentry PUBLIC
"-//OASIS//DTD DocBook XML V4.1.2//EN"
"docbook/docbookx.dtd">
<refentry id='irk.8'>
<refmeta>
<refentrytitle>irk</refentrytitle>
<manvolnum>1</manvolnum>
<refmiscinfo class='date'>Apr 30 2014</refmiscinfo>
<refmiscinfo class='source'>irker</refmiscinfo>
<refmiscinfo class='product'>irker</refmiscinfo>
<refmiscinfo class='manual'>Commands</refmiscinfo>
</refmeta>
<refnamediv id='name'>
<refname>irk</refname>
<refpurpose>test program for irkerd</refpurpose>
</refnamediv>
<refsynopsisdiv id='synopsis'>
<cmdsynopsis>
<command>irk</command>
<arg><replaceable>target</replaceable></arg>
<arg choice='opt'><replaceable>message text</replaceable></arg>
</cmdsynopsis>
</refsynopsisdiv>
<refsect1 id='description'><title>DESCRIPTION</title>
<para><application>irk</application> is a simple test program for
<citerefentry><refentrytitle>irkerd</refentrytitle><manvolnum>8</manvolnum></citerefentry>. It
will construct a simple JSON object and pass it to the daemon running
on localhost.</para>
</refsect1>
<refsect1 id='options'><title>OPTIONS</title>
<para><application>irk</application> takes the following options:</para>
<variablelist>
<varlistentry>
<term>target</term>
<listitem><para>Which server and channel to join to announced the
message. If not prefixed with "irc:", it will prefix
"irc://chat.freenode.net/" to the argument before passing it directly
to irkerd. This argument is passed as the "to" parameter in the JSON
object.</para></listitem>
</varlistentry>
<varlistentry>
<term>message</term>
<listitem><para>Which message to send to the target specified
above. If the string "-", the message will be read from standard
input, with newlines stripped.</para></listitem>
</varlistentry>
</variablelist>
</refsect1>
<refsect1 id='limitations'><title>LIMITATIONS</title>
<para><application>irk</application> has no commandline usage and may
be riddled with bugs.</para>
<para><application>irk</application> doesn't know how to talk to your
favorite VCS. You will generally want to use
<citerefentry><refentrytitle>irkerhook</refentrytitle><manvolnum>1</manvolnum></citerefentry>
instead</para>
<para><application>irk</application> has also all the limitations of
<application>irkerd</application>.</para>
</refsect1>
<refsect1 id='see_also'><title>SEE ALSO</title>
<para>
<citerefentry><refentrytitle>irkerhook</refentrytitle><manvolnum>1</manvolnum></citerefentry>,
</para>
</refsect1>
<refsect1 id='authors'><title>AUTHOR</title>
<para>Eric S. Raymond <email>esr@snark.thyrsus.com</email>. See the
project page at <ulink
url='http://www.catb.org/~esr/irker'>http://www.catb.org/~esr/irker</ulink>
for updates and other resources, including an installable repository
hook script.</para>
</refsect1>
</refentry>

1147
irkerd Executable file
View file

File diff suppressed because it is too large Load diff

16
irkerd.service Normal file
View file

@ -0,0 +1,16 @@
# Copyright 2012 Wulf C. Krueger <philantrop@exherbo.org>
# Distributed under the terms of the BSD LICENSE
[Unit]
Description=Internet Relay Chat (IRC) notification daemon
Requires=network.target
Documentation=man:irkerd(8) man:irkerhook(1) man:irk(1)
[Service]
User=irker
ExecStart=/usr/bin/irkerd
User=irker
[Install]
WantedBy=multi-user.target
Alias=irker.service

261
irkerd.xml Normal file
View file

@ -0,0 +1,261 @@
<!DOCTYPE refentry PUBLIC
"-//OASIS//DTD DocBook XML V4.1.2//EN"
"docbook/docbookx.dtd">
<refentry id='irkerd.8'>
<refmeta>
<refentrytitle>irkerd</refentrytitle>
<manvolnum>8</manvolnum>
<refmiscinfo class='date'>Aug 27 2012</refmiscinfo>
<refmiscinfo class='source'>irker</refmiscinfo>
<refmiscinfo class='product'>irker</refmiscinfo>
<refmiscinfo class='manual'>Commands</refmiscinfo>
</refmeta>
<refnamediv id='name'>
<refname>irkerd</refname>
<refpurpose>relay for shipping notifications to IRC servers</refpurpose>
</refnamediv>
<refsynopsisdiv id='synopsis'>
<cmdsynopsis>
<command>irkerd</command>
<arg>-c <replaceable>ca-file</replaceable></arg>
<arg>-d <replaceable>debuglevel</replaceable></arg>
<arg>-e <replaceable>cert-file</replaceable></arg>
<arg>-l <replaceable>logfile</replaceable></arg>
<arg>-H <replaceable>host</replaceable></arg>
<arg>-n <replaceable>nick</replaceable></arg>
<arg>-p <replaceable>password</replaceable></arg>
<arg>-P <replaceable>password-file</replaceable></arg>
<arg>-i <replaceable>IRC-URL</replaceable></arg>
<arg>-t <replaceable>timeout</replaceable></arg>
<arg>-V</arg>
<arg>-h</arg>
<arg choice='opt'><replaceable>message text</replaceable></arg>
</cmdsynopsis>
</refsynopsisdiv>
<refsect1 id='description'><title>DESCRIPTION</title>
<para><application>irkerd</application> is a specialized write-only IRC
client intended to be used for shipping notification messages to IRC
channels. The use case in mind when it was designed was broadcasting
notifications from commit hooks in version-control systems.</para>
<para>The main advantage of relaying through this daemon over
individual scripted sends from applications is that it can maintain
connection state for multiple channels, rather than producing obnoxious
join/leave channel spam on every message.</para>
<para><application>irkerd</application> is a socket server that
listens on for UDP or TCP packets on port 6659 for textual request
lines containing JSON objects and terminated by a newline. Each JSON
object must have two members: "to" specifying a destination or
destination list, and "privmsg" specifying the message text.
Examples:
<programlisting>
{"to":"irc://chat.freenode.net/git-ciabot", "privmsg":"Hello, world!"}
{"to":["irc://chat.freenode.net/#git-ciabot","irc://chat.freenode.net/#gpsd"],"privmsg":"Multichannel test"}
{"to":"irc://chat.hypothetical.net:6668/git-ciabot", "privmsg":"Hello, world!"}
{"to":"ircs://chat.hypothetical.net/git-private?key=topsecret", "privmsg":"Keyed channel test"}
{"to":"ircs://:topsecret@chat.example.net/git-private", "privmsg":"Password-protected server test"}
</programlisting></para>
<para>If the channel part of the URL does not have one of the prefix
characters <quote>#</quote>, <quote>&amp;</quote>, or
<quote>+</quote>, a <quote>#</quote> will be prepended to it before
shipping - <emphasis>unless</emphasis> the channel part has the suffix
",isnick" (which is unconditionally removed).</para>
<para>The host part of the URL may have a port-number suffix separated by a
colon, as shown in the third example; otherwise
<application>irkerd</application> sends plaintext messages to the default
6667 IRC port of each server, and SSL/TLS messages to 6697.</para>
<para>The password for password-protected servers can be set using the
usual <quote>[{username}:{password}@]{host}:{port}</quote> defined in
RFC 3986, as shown in the fifth example. Non-empty URL usernames
override the default <quote>irker</quote> username.</para>
<para>When the <quote>to</quote> URL uses the <quote>ircs</quote>
scheme (as shown in the fourth and fifth examples), the connection to
the IRC server is made via SSL/TLS (vs. a plaintext connection with the
<quote>irc</quote> scheme). To connect via SSL/TLS with Python 2.x,
you need to explicitly declare the certificate authority file used to
verify server certificates. For example, <quote>-c
/etc/ssl/certs/ca-certificates.crt</quote>. In Python 3.2 and later,
you can still set this option to declare a custom CA file, but
<application>irkerd</application>; if you don't set it
<application>irkerd</application> will use OpenSSL's default file
(using Python's
<quote>ssl.SSLContext.set_default_verify_paths</quote>). In Python
3.2 and later, <quote>ssl.match_hostname</quote> is used to ensure the
server certificate belongs to the intended host, as well as being
signed by a trusted CA.</para>
<para>To join password-protected (mode +k) channels, the channel part of the
URL may be followed with a query-string indicating the channel key, of the
form <quote>?secret</quote> or <quote>?key=secret</quote>, where
<quote>secret</quote> is the channel key.</para>
<para>An empty message is legal and will cause
<application>irkerd</application> to join or maintain a connection to
the target channels without actually emitting a message. This may be
useful for advertising that an instance is up and running, or for
joining a channel to log its traffic.</para>
</refsect1>
<refsect1 id='options'><title>OPTIONS</title>
<para><application>irkerd</application> takes the following options:</para>
<variablelist>
<varlistentry>
<term>-d</term>
<listitem>
<para>
Takes a following value, setting the debugging level from it;
possible values are 'critical', 'error', 'warning', 'info',
'debug'. This option will generally only be of interest to
developers, as the logs are designed to help trace
<application>irkerd</application>'s internal state. These tracing
logs are independent of the traffic logs controlled by
<quote>-l</quote>.
</para>
<para>
Logging will be to standard error (if
<application>irkerd</application> is running in the foreground) or
to <quote>/dev/syslog</quote> with facility "daemon" (if
<application>irkerd</application> is running in the background).
The background-ness of <application>irkerd</application> is
determined by comparing the process group id with the process
group associated with the terminal attached to stdout (with
non-matches for background processes). We assume you aren't
running <application>irkerd</application> in Windows or another OS
that doesn't support <quote>os.getpgrp</quote> or
<quote>tcgetpgrp</quote>. We assume that if stdout is attached to
a TTY associated with the same process group as
<application>irkerd</application>, you do intend to log to stderr
and not syslog.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>-e</term>
<listitem><para>Takes a following filename in pem format and uses it
to authenticate to the IRC server. You must be connecting to the IRC server
over SSL for this to function properly. This is commonly known as
<quote>CertFP.</quote>
</para></listitem>
</varlistentry>
<varlistentry>
<term>-e</term>
<listitem><para>Takes a following filename in pem format and uses it
to authenticate to the IRC server. You must be connecting to the IRC
server over SSL for this to function properly. This is commonly known
as <quote>CertFP.</quote></para>
</listitem>
</varlistentry>
<varlistentry>
<term>-l</term>
<listitem><para>Takes a following filename, logs traffic to that file.
Each log line consists of three |-separated fields; a numeric
timestamp in Unix time, the FQDN of the sending server, and the
message data.</para></listitem>
</varlistentry>
<varlistentry>
<term>-H</term>
<listitem><para>Takes a following hostname, and binds to that address
when listening for messages. <application>irkerd</application> binds
to localhost by default, but you may want to use your host's public
address to listen on a local network. Listening on a public interface
is not recommended, as it makes spamming IRC channels very
easy.</para></listitem>
</varlistentry>
<varlistentry>
<term>-n</term>
<listitem><para>Takes a following value, setting the nick
to be used. If the nick contains a numeric format element
(such as %03d) it is used to generate suffixed fallback names
in the event of a nick collision.</para></listitem>
</varlistentry>
<varlistentry>
<term>-p</term>
<listitem><para>Takes a following value, setting a nickserv
password to be used. If given, this password is shipped to
authenticate the nick on receipt of a welcome message.</para></listitem>
</varlistentry>
<varlistentry>
<term>-P</term>
<listitem><para>Liuke p, but the argument is interpreted as a filename
from which to read the password</para></listitem>
</varlistentry>
<varlistentry>
<term>-t</term>
<listitem><para>Takes a following value, setting the connection
timeout for server-socket opens.</para></listitem>
</varlistentry>
<varlistentry>
<term>-i</term>
<listitem><para>Immediate mode, to be run in foreground. Takes a following
following value interpreted as a channel URL. May take a second
argument giving a message string; if the second argument is absent the
message is read from standard input (and may contain newlines).
Sends the message, then quits.</para></listitem>
</varlistentry>
<varlistentry>
<term>-V</term>
<listitem><para>Write the program version to stdout and
terminate.</para></listitem>
</varlistentry>
<varlistentry>
<term>-h</term>
<listitem><para>Print usage instructions and terminate.</para></listitem>
</varlistentry>
</variablelist>
</refsect1>
<refsect1 id='limitations'><title>LIMITATIONS</title>
<para>Requests via UDP optimizes for lowest latency and network load
by avoiding TCP connection setup time; the cost is that delivery is
not reliable in the face of packet loss.</para>
<para>An <application>irkerd</application> instance with a
publicly-accessible request socket could complicate blocking of IRC
spam by making it easy for spammers to submit while hiding their IP
addresses; the better way to deploy, then, is on places like
project-hosting sites where the <application>irkerd</application>
socket can be visible from commit-hook code but not exposed to the
outside world. Priming your firewall with blocklists of IP addresses
known to spew spam is always a good idea.</para>
<para>The absence of any option to set the service port is deliberate.
If you think you need to do that, you have a problem better solved at
your firewall.</para>
<para>IRC has a message length limit of 510 bytes; generate your
privmsg attribute values with appropriate care.</para>
<para>IRC ignores any text after an embedded newline. Be aware that
<application>irkerd</application> will turn payload strings with
embedded newlines into multiple IRC sends to avoid having message data
discarded. </para>
<para>Due to a bug in Python URL parsing, IRC urls with both a # and a
key part may fail unexpectedly. The workaround is to remove the #.</para>
</refsect1>
<refsect1 id='see_also'><title>SEE ALSO</title>
<para>
<citerefentry><refentrytitle>irkerhook</refentrytitle><manvolnum>1</manvolnum></citerefentry>,
</para>
</refsect1>
<refsect1 id='authors'><title>AUTHOR</title>
<para>Eric S. Raymond <email>esr@snark.thyrsus.com</email>. See the
project page at <ulink
url='http://www.catb.org/~esr/irker'>http://www.catb.org/~esr/irker</ulink>
for updates and other resources, including an installable repository
hook script.</para>
</refsect1>
</refentry>

609
irkerhook.py Executable file
View file

@ -0,0 +1,609 @@
#!/usr/bin/env python3
# Copyright (c) 2012 Eric S. Raymond <esr@thyrsus.com>
# SPDX-License-Identifier: BSD-2-Clause
'''
This script contains git porcelain and porcelain byproducts.
Requires either Python 2.6, or 2.5 with the simplejson library installed
or Python 3.x.
usage: irkerhook.py [-V] [-n] [--variable=value...] [commit_id...]
This script is meant to be run in an update or post-commit hook.
Try it with -n to see the notification dumped to stdout and verify
that it looks sane. With -V this script dumps its version and exits.
See the irkerhook manual page in the distribution for a detailed
explanation of how to configure this hook.
The default location of the irker proxy, if the project configuration
does not override it.
'''
# SPDX-License-Identifier: BSD-2-Clause
from __future__ import print_function, absolute_import
# pylint: disable=line-too-long,invalid-name,missing-function-docstring,missing-class-docstring,no-else-break,no-else-return,too-many-instance-attributes,too-many-locals,too-many-branches,too-many-statements,redefined-outer-name,import-outside-toplevel,raise-missing-from
default_server = "localhost"
IRKER_PORT = 6659
# The default service used to turn your web-view URL into a tinyurl so it
# will take up less space on the IRC notification line.
default_tinyifier = u"http://tinyurl.com/api-create.php?url="
# Map magic urlprefix values to actual URL prefixes.
urlprefixmap = {
"viewcvs": "http://%(host)s/viewcvs/%(repo)s?view=revision&revision=",
"gitweb": "http://%(host)s/cgi-bin/gitweb.cgi?p=%(repo)s;a=commit;h=",
"cgit": "http://%(host)s/cgi-bin/cgit.cgi/%(repo)s/commit/?id=",
}
# By default, ship to the freenode #commits list
default_channels = u"irc://chat.freenode.net/#commits"
#
# No user-serviceable parts below this line:
#
version = "2.21"
# pylint: disable=multiple-imports,wrong-import-position
import os, sys, socket, subprocess, locale, datetime, re
try
from shlex import quote as shellquote
except ImportError:
from pipes import quote as shellquote
try:
from urllib2 import urlopen, HTTPError
except ImportError:
from urllib.error import HTTPError
from urllib.request import urlopen
try:
import simplejson as json # Faster, also makes us Python-2.5-compatible
except ImportError:
import json
if sys.version_info.major == 2:
# pylint: disable=undefined-variable
string_type = unicode
else:
string_type = str
try:
getstatusoutput = subprocess.getstatusoutput
except AttributeError:
# pylint: disable=import-error
import commands
getstatusoutput = commands.getstatusoutput
def do(command):
if sys.version_info.major == 2:
return string_type(getstatusoutput(command)[1], locale.getlocale()[1] or 'UTF-8')
else:
return getstatusoutput(command)[1]
# pylint: disable=too-few-public-methods
class Commit:
def __init__(self, extractor, commit):
"Per-commit data."
self.commit = commit
self.branch = None
self.rev = None
self.mail = None
self.author = None
self.files = None
self.logmsg = None
self.url = None
self.author_name = None
self.author_date = None
self.commit_date = None
self.id = None
self.__dict__.update(extractor.__dict__)
if sys.version_info.major == 2:
# Convert __str__ to __unicode__ for python 2
self.__unicode__ = self.__str__
# Not really needed, but maybe useful for debugging
self.__str__ = lambda x: x.__unicode__().encode('utf-8')
def __str__(self):
"Produce a notification string from this commit."
# pylint: disable=no-member
if not self.urlprefix:
self.url = ""
else:
# pylint: disable=no-member
urlprefix = urlprefixmap.get(self.urlprefix, self.urlprefix)
webview = (urlprefix % self.__dict__) + self.commit
try:
# See it the url is accessible
res = urlopen(webview)
if self.tinyifier and self.tinyifier.lower() != "none":
try:
# Didn't get a retrieval error on the web
# view, so try to tinyify a reference to it.
self.url = urlopen(self.tinyifier + webview).read()
try:
self.url = self.url.decode('UTF-8')
except UnicodeError:
pass
except IOError:
self.url = webview
else:
self.url = webview
except HTTPError as e:
if e.code == 401:
# Authentication error, so we assume the view is valid
self.url = webview
else:
self.url = ""
except IOError:
self.url = ""
# pylint: disable=no-member
res = self.template % self.__dict__
return string_type(res, 'UTF-8') if not isinstance(res, string_type) else res
class GenericExtractor:
"Generic class for encapsulating data from a VCS."
booleans = ["tcp"]
numerics = ["maxchannels"]
strings = ["email"]
def __init__(self, arguments):
self.arguments = arguments
self.project = None
self.repo = None
# These aren't really repo data but they belong here anyway...
self.email = None
self.tcp = True
self.tinyifier = default_tinyifier
self.server = None
self.channels = None
self.maxchannels = 0
self.template = None
self.urlprefix = None
self.host = socket.getfqdn()
self.cialike = None
self.filtercmd = None
# Color highlighting is disabled by default.
self.color = None
self.bold = self.green = self.blue = self.yellow = self.red = ""
self.brown = self.magenta = self.cyan = self.reset = ""
def activate_color(self, style):
"IRC color codes."
if style == 'mIRC':
# mIRC colors are mapped as closely to the ANSI colors as
# possible. However, bright colors (green, blue, red,
# yellow) have been made their dark counterparts since
# ChatZilla does not properly darken mIRC colors in the
# Light Motif color scheme.
self.bold = '\x02'
self.green = '\x0303'
self.blue = '\x0302'
self.red = '\x0304'
self.red = '\x0305'
self.yellow = '\x0307'
self.brown = '\x0305'
self.magenta = '\x0306'
self.cyan = '\x0310'
self.reset = '\x0F'
if style == 'ANSI':
self.bold = '\x1b[1m'
self.green = '\x1b[1;32m'
self.blue = '\x1b[1;34m'
self.red = '\x1b[1;31m'
self.yellow = '\x1b[1;33m'
self.brown = '\x1b[33m'
self.magenta = '\x1b[35m'
self.cyan = '\x1b[36m'
self.reset = '\x1b[0m'
def load_preferences(self, conf):
"Load preferences from a file in the repository root."
if not os.path.exists(conf):
return
ln = 0
for line in open(conf):
ln += 1
if line.startswith("#") or not line.strip():
continue
if line.count('=') != 1:
sys.stderr.write('%s:%d: missing = in config line\n' \
% (conf, ln))
continue
fields = line.split('=')
if len(fields) != 2:
sys.stderr.write('%s:%d: too many fields in config line\n' \
% (conf, ln))
continue
variable = fields[0].strip()
value = fields[1].strip()
if value.lower() == "true":
value = True
elif value.lower() == "false":
value = False
# User cannot set maxchannels - only a command-line arg can do that.
if variable == "maxchannels":
return
setattr(self, variable, value)
def do_overrides(self):
"Make command-line overrides possible."
for tok in self.arguments:
for key in self.__dict__:
if tok.startswith("--" + key + "="):
val = tok[len(key)+3:]
setattr(self, key, val)
for (key, val) in self.__dict__.items():
if key in GenericExtractor.booleans:
if isinstance(val, str) and val.lower() == "true":
setattr(self, key, True)
elif isinstance(val, str) and val.lower() == "false":
setattr(self, key, False)
elif key in GenericExtractor.numerics:
setattr(self, key, int(val))
elif key in GenericExtractor.strings:
setattr(self, key, val)
if not self.project:
sys.stderr.write("irkerhook.py: no project name set!\n")
raise SystemExit(1)
if not self.repo:
self.repo = self.project.lower()
if not self.channels:
self.channels = default_channels % self.__dict__
if self.color and self.color.lower() != "none":
self.activate_color(self.color)
def has(dirname, paths):
"Test for existence of a list of paths."
# all() is a python2.5 construct
for exists in [os.path.exists(os.path.join(dirname, x)) for x in paths]:
if not exists:
return False
return True
# VCS-dependent code begins here
class GitExtractor(GenericExtractor):
"Metadata extraction for the git version control system."
@staticmethod
def is_repository(dirname):
# Must detect both ordinary and bare repositories
return has(dirname, [".git"]) or \
has(dirname, ["HEAD", "refs", "objects"])
def __init__(self, arguments):
GenericExtractor.__init__(self, arguments)
# Get all global config variables
self.project = do("git config --get irker.project")
self.repo = do("git config --get irker.repo")
self.server = do("git config --get irker.server")
self.channels = do("git config --get irker.channels")
self.email = do("git config --get irker.email")
self.tcp = do("git config --bool --get irker.tcp")
self.template = do("git config --get irker.template") or u'%(bold)s%(project)s:%(reset)s %(green)s%(author)s%(reset)s %(repo)s:%(yellow)s%(branch)s%(reset)s * %(bold)s%(rev)s%(reset)s / %(bold)s%(files)s%(reset)s: %(logmsg)s %(brown)s%(url)s%(reset)s'
self.tinyifier = do("git config --get irker.tinyifier") or default_tinyifier
self.color = do("git config --get irker.color")
self.urlprefix = do("git config --get irker.urlprefix") or u"gitweb"
self.cialike = do("git config --get irker.cialike")
self.filtercmd = do("git config --get irker.filtercmd")
# These are git-specific
self.refname = do("git symbolic-ref HEAD 2>/dev/null")
self.revformat = do("git config --get irker.revformat")
# The project variable defaults to the name of the repository toplevel.
if not self.project:
bare = do("git config --bool --get core.bare")
if bare.lower() == "true":
keyfile = "HEAD"
else:
keyfile = ".git/HEAD"
here = os.getcwd()
while True:
if os.path.exists(os.path.join(here, keyfile)):
self.project = os.path.basename(here)
if self.project.endswith('.git'):
self.project = self.project[0:-4]
break
elif here == '/':
sys.stderr.write("irkerhook.py: no git repo below root!\n")
sys.exit(1)
here = os.path.dirname(here)
# Get overrides
self.do_overrides()
# pylint: disable=no-self-use
def head(self):
"Return a symbolic reference to the tip commit of the current branch."
return "HEAD"
def commit_factory(self, commit_id):
"Make a Commit object holding data for a specified commit ID."
commit = Commit(self, commit_id)
commit.branch = re.sub(r"^refs/[^/]*/", "", self.refname)
# Compute a description for the revision
if self.revformat == 'raw':
commit.rev = commit.commit
elif self.revformat == 'short':
commit.rev = ''
else: # self.revformat == 'describe'
commit.rev = do("git describe %s 2>/dev/null" % shellquote(commit.commit))
if not commit.rev:
# Query git for the abbreviated hash
commit.rev = do("git log -1 '--pretty=format:%h' " + shellquote(commit.commit))
if self.urlprefix in ('gitweb', 'cgit'):
# Also truncate the commit used for the announced urls
commit.commit = commit.rev
# Extract the meta-information for the commit
commit.files = do("git diff-tree -r --name-only " + shellquote(commit.commit))
commit.files = " ".join(commit.files.strip().split("\n")[1:])
# Design choice: for git we ship only the first message line, which is
# conventionally supposed to be a summary of the commit. Under
# other VCSes a different choice may be appropriate.
commit.author_name, commit.mail, commit.logmsg = \
do("git log -1 '--pretty=format:%an%n%ae%n%s' " + shellquote(commit.commit)).split("\n")
# This discards the part of the author's address after @.
# Might be be nice to ship the full email address, if not
# for spammers' address harvesters - getting this wrong
# would make the freenode #commits channel into harvester heaven.
commit.author = commit.mail.split("@")[0]
commit.author_date, commit.commit_date = \
do("git log -1 '--pretty=format:%ai|%ci' " + shellquote(commit.commit)).split("|")
return commit
class SvnExtractor(GenericExtractor):
"Metadata extraction for the svn version control system."
@staticmethod
def is_repository(dirname):
return has(dirname, ["format", "hooks", "locks"])
def __init__(self, arguments):
GenericExtractor.__init__(self, arguments)
# Some things we need to have before metadata queries will work
self.repository = '.'
for tok in arguments:
if tok.startswith("--repository="):
self.repository = tok[13:]
self.project = os.path.basename(self.repository)
self.template = '%(bold)s%(project)s%(reset)s: %(green)s%(author)s%(reset)s %(repo)s * %(bold)s%(rev)s%(reset)s / %(bold)s%(files)s%(reset)s: %(logmsg)s %(brown)s%(url)s%(reset)s'
self.urlprefix = "viewcvs"
self.id = None
self.load_preferences(os.path.join(self.repository, "irker.conf"))
self.do_overrides()
# pylint: disable=no-self-use
def head(self):
sys.stderr.write("irker: under svn, hook requires a commit argument.\n")
raise SystemExit(1)
def commit_factory(self, commit_id):
self.id = commit_id
commit = Commit(self, commit_id)
commit.branch = ""
commit.rev = "r%s" % self.id
commit.author = self.svnlook("author")
commit.commit_date = self.svnlook("date").partition('(')[0]
commit.files = self.svnlook("dirs-changed").strip().replace("\n", " ")
commit.logmsg = self.svnlook("log").strip()
return commit
def svnlook(self, info):
return do("svnlook %s %s --revision %s" % (shellquote(info), shellquote(self.repository), shellquote(self.id)))
class HgExtractor(GenericExtractor):
"Metadata extraction for the Mercurial version control system."
@staticmethod
def is_repository(directory):
return has(directory, [".hg"])
def __init__(self, arguments):
from mercurial.encoding import unifromlocal, unitolocal
# This fiddling with arguments is necessary since the Mercurial hook can
# be run in two different ways: either directly via Python (in which
# case hg should be pointed to the hg_hook function below) or as a
# script (in which case the normal __main__ block at the end of this
# file is exercised). In the first case, we already get repository and
# ui objects from Mercurial, in the second case, we have to create them
# from the root path.
self.repository = None
if arguments and isinstance(arguments[0], tuple):
# Called from hg_hook function
ui, self.repository = arguments[0]
arguments = [] # Should not be processed further by do_overrides
else:
# Called from command line: create repo/ui objects
from mercurial import hg, ui as uimod
repopath = b'.'
for tok in arguments:
if tok.startswith('--repository='):
repopath = unitolocal(tok[13:])
ui = uimod.ui()
ui.readconfig(os.path.join(repopath, b'.hg', b'hgrc'), repopath)
self.repository = hg.repository(ui, repopath)
GenericExtractor.__init__(self, arguments)
# Extract global values from the hg configuration file(s)
self.project = unifromlocal(ui.config(b'irker', b'project') or b'')
self.repo = unifromlocal(ui.config(b'irker', b'repo') or b'')
self.server = unifromlocal(ui.config(b'irker', b'server') or b'')
self.channels = unifromlocal(ui.config(b'irker', b'channels') or b'')
self.email = unifromlocal(ui.config(b'irker', b'email') or b'')
self.tcp = str(ui.configbool(b'irker', b'tcp')) # converted to bool again in do_overrides
self.template = unifromlocal(ui.config(b'irker', b'template') or b'')
if not self.template:
self.template = '%(bold)s%(project)s:%(reset)s %(green)s%(author)s%(reset)s %(repo)s:%(yellow)s%(branch)s%(reset)s * %(bold)s%(rev)s%(reset)s / %(bold)s%(files)s%(reset)s: %(logmsg)s %(brown)s%(url)s%(reset)s'
self.tinyifier = unifromlocal(ui.config(
b'irker', b'tinyifier',
default=default_tinyifier.encode('utf-8')))
self.color = unifromlocal(ui.config(b'irker', b'color') or b'')
self.urlprefix = unifromlocal(ui.config(
b'irker', b'urlprefix', default=ui.config(b'web', b'baseurl')))
if self.urlprefix:
# self.commit is appended to this by do_overrides
self.urlprefix = (
self.urlprefix.rstrip('/')
+ '/%s/rev/' % unifromlocal(self.repository.root).rstrip('/'))
self.cialike = unifromlocal(ui.config(b'irker', b'cialike') or b'')
self.filtercmd = unifromlocal(ui.config(b'irker', b'filtercmd') or b'')
if not self.project:
self.project = os.path.basename(unifromlocal(self.repository.root).rstrip('/'))
self.do_overrides()
# pylint: disable=no-self-use
def head(self):
"Return a symbolic reference to the tip commit of the current branch."
return "-1"
def commit_factory(self, commit_id):
"Make a Commit object holding data for a specified commit ID."
from mercurial.node import short
from mercurial.templatefilters import person
from mercurial.encoding import unifromlocal, unitolocal
if isinstance(commit_id, str) and not isinstance(commit_id, bytes):
commit_id = unitolocal(commit_id)
ctx = self.repository[commit_id]
commit = Commit(self, unifromlocal(short(ctx.hex())))
# Extract commit-specific values from a "context" object
commit.rev = '%d:%s' % (ctx.rev(), commit.commit)
commit.branch = unifromlocal(ctx.branch())
commit.author = unifromlocal(person(ctx.user()))
commit.author_date = \
datetime.datetime.fromtimestamp(ctx.date()[0]).strftime('%Y-%m-%d %H:%M:%S')
commit.logmsg = unifromlocal(ctx.description())
# Extract changed files from status against first parent
st = self.repository.status(ctx.p1().node(), ctx.node())
commit.files = unifromlocal(b' '.join(st.modified + st.added + st.removed))
return commit
def hg_hook(ui, repo, **kwds):
# To be called from a Mercurial "commit", "incoming" or "changegroup" hook.
# Example configuration:
# [hooks]
# incoming.irker = python:/path/to/irkerhook.py:hg_hook
extractor = HgExtractor([(ui, repo)])
start = repo[kwds['node']].rev()
end = len(repo)
if start != end:
# changegroup with multiple commits, so we generate a notification
# for each one
for rev in range(start, end):
ship(extractor, rev, False)
else:
ship(extractor, kwds['node'], False)
# The files we use to identify a Subversion repo might occur as content
# in a git or hg repo, but the special subdirectories for those are more
# reliable indicators. So test for Subversion last.
extractors = [GitExtractor, HgExtractor, SvnExtractor]
# VCS-dependent code ends here
def convert_message(message):
"""Convert the message to bytes to send to the socket"""
return message.encode(locale.getlocale()[1] or 'UTF-8') + b'\n'
def ship(extractor, commit, debug):
"Ship a notification for the specified commit."
metadata = extractor.commit_factory(commit)
# This is where we apply filtering
if extractor.filtercmd:
cmd = '%s %s' % (shellquote(extractor.filtercmd),
shellquote(json.dumps(metadata.__dict__)))
data = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE).stdout.read()
try:
metadata.__dict__.update(json.loads(data))
except ValueError:
sys.stderr.write("irkerhook.py: could not decode JSON: %s\n" % data)
raise SystemExit(1)
# Rewrite the file list if too long. The objective here is only
# to be easier on the eyes.
if extractor.cialike \
and extractor.cialike.lower() != "none" \
and len(metadata.files) > int(extractor.cialike):
files = metadata.files.split()
dirs = {d.rpartition('/')[0] for d in files}
if len(dirs) == 1:
metadata.files = "(%s files)" % (len(files),)
else:
metadata.files = "(%s files in %s dirs)" % (len(files), len(dirs))
# Message reduction. The assumption here is that IRC can't handle
# lines more than 510 characters long. If we exceed that length, we
# try knocking out the file list, on the theory that for notification
# purposes the commit text is more important. If it's still too long
# there's nothing much can be done other than ship it expecting the IRC
# server to truncate.
privmsg = string_type(metadata)
if len(privmsg) > 510:
metadata.files = ""
privmsg = string_type(metadata)
# Anti-spamming guard. It's deliberate that we get maxchannels not from
# the user-filtered metadata but from the extractor data - means repo
# administrators can lock in that setting.
channels = metadata.channels.split(",")
if extractor.maxchannels != 0:
channels = channels[:extractor.maxchannels]
# Ready to ship.
message = json.dumps({"to": channels, "privmsg": privmsg})
if debug:
print(message)
elif channels:
try:
if extractor.email:
# We can't really figure out what our SF username is without
# exploring our environment. The mail pipeline doesn't care
# about who sent the mail, other than being from sourceforge.
# A better way might be to simply call mail(1)
sender = "irker@users.sourceforge.net"
msg = """From: %(sender)s
Subject: irker json
%(message)s""" % {"sender":sender, "message":message}
import smtplib
smtp = smtplib.SMTP()
smtp.connect()
smtp.sendmail(sender, extractor.email, msg)
smtp.quit()
elif extractor.tcp:
try:
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((extractor.server or default_server, IRKER_PORT))
sock.sendall(convert_message(message))
finally:
sock.close()
else:
try:
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.sendto(convert_message(message), (extractor.server or default_server, IRKER_PORT))
finally:
sock.close()
except socket.error as e:
sys.stderr.write("%s\n" % e)
if __name__ == "__main__":
notify = True
repository = os.getcwd()
commits = []
for arg in sys.argv[1:]:
if arg == '-n':
notify = False
elif arg == '-V':
print("irkerhook.py: version", version)
sys.exit(0)
elif arg.startswith("--repository="):
repository = arg[13:]
elif not arg.startswith("--"):
commits.append(arg)
# Figure out which extractor we should be using
for candidate in extractors:
if candidate.is_repository(repository):
cls = candidate
break
else:
sys.stderr.write("irkerhook: cannot identify a repository type.\n")
raise SystemExit(1)
extractor = cls(sys.argv[1:])
# And apply it.
if not commits:
commits = [extractor.head()]
for commit in commits:
ship(extractor, commit, not notify)
# The following sets edit modes for GNU EMACS
# Local Variables:
# mode:python
# End:

417
irkerhook.xml Normal file
View file

@ -0,0 +1,417 @@
<!DOCTYPE refentry PUBLIC
"-//OASIS//DTD DocBook XML V4.1.2//EN"
"docbook/docbookx.dtd">
<refentry id='irkerhook.1'>
<refmeta>
<refentrytitle>irkerhook</refentrytitle>
<manvolnum>1</manvolnum>
<refmiscinfo class='date'>Aug 27 2012</refmiscinfo>
<refmiscinfo class='source'>irker</refmiscinfo>
<refmiscinfo class='product'>irker</refmiscinfo>
<refmiscinfo class='manual'>Commands</refmiscinfo>
</refmeta>
<refnamediv id='name'>
<refname>irkerhook</refname>
<refpurpose>repository hook script issuing irker notifications</refpurpose>
</refnamediv>
<refsynopsisdiv id='synopsis'>
<cmdsynopsis>
<command>irkerhook.py</command>
<arg>-n</arg>
<arg>-V</arg>
<group><arg rep='repeat'><replaceable>--variable=value</replaceable></arg></group>
<group><arg rep='repeat'><replaceable>commit-id</replaceable></arg></group>
</cmdsynopsis>
</refsynopsisdiv>
<refsect1 id='description'><title>DESCRIPTION</title>
<para><application>irkerhook.py</application> is a Python script intended
to be called from the post-commit hook of a version-control repository. Its
job is to collect information about the commit that fired the hook (and
possibly preferences set by the repository owner) and ship that information
to an instance of <application>irkerd</application> for forwarding to
various announcement channels.</para>
<para>The proper invocation and behavior of
<application>irkerhook.py</application> varies depending on which
VCS (version-control system) is calling it. There are four different places
from which it may extract information:</para>
<orderedlist>
<listitem><para>Calls to VCS utilities.</para></listitem>
<listitem><para>In VCSes like git that support user-settable configuration
variables, variables with the prefix "irker.".</para></listitem>
<listitem><para>In other VCSes, a configuration file, "irker.conf", in the
repository's internals directory.</para></listitem>
<listitem><para>Command-line arguments of the form
--variable=value.</para></listitem>
</orderedlist>
<para>The following variables are general to all supported VCSes:</para>
<variablelist>
<varlistentry>
<term>project</term>
<listitem>
<para>The name of the project. Should be a relatively short identifier;
will usually appear at the very beginning of a notification.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>repo</term>
<listitem>
<para>The name of the repository top-level directory. If not
specified, defaults to a lowercased copy of the project name.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>channels</term>
<listitem>
<para>An IRC channel URL, or comma-separated list of same, identifying
channels to which notifications are to be sent. If not specified, the
default is the freenode #commits channel.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>server</term>
<listitem>
<para>The host on which the notification-relaying irker daemon is expected
to reside. Defaults to "localhost".</para>
</listitem>
</varlistentry>
<varlistentry>
<term>email</term>
<listitem>
<para>If set, use email for communication rather than TCP or UDP.
The value is used as the target mail address.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>tcp</term>
<listitem>
<para>If "true", use TCP for communication; if "false", use UDP.
Defaults to "false".</para>
</listitem>
</varlistentry>
<varlistentry>
<term>urlprefix</term>
<listitem>
<para>Changeset URL prefix for your repo. When the commit ID is appended
to this, it should point at a CGI that will display the commit
through cgit, gitweb or something similar. The defaults will probably
work if you have a typical gitweb/cgit setup.</para>
<para>If the value of this variable is "None", generation of the URL
field in commit notifications will be suppressed. Other magic values
are "cgit", "gitweb", and "viewcvs", which expand to URL templates
that will usually work with those systems.</para>
<para>The magic cookies "%(host)s" and %(repo)s" may occur in this
URL. The former is expanded to the FQDN of the host on which
<application>irkerhook.py</application> is running; the latter is
expanded to the value of the "repo" variable.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>tinyifier</term>
<listitem>
<para>URL template pointing to a service for compressing URLs so they
will take up less space in the notification line. If the value of this
variable is "None", no compression will be attempted.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>color</term>
<listitem>
<para>If "mIRC", highlight notification fields with mIRC color codes.
If "ANSI", highlight notification fields with ANSI color escape
sequences. Defaults to "none" (no colors). ANSI codes are supported
in Chatzilla, irssi, ircle, and BitchX; mIRC codes only are recognized
in mIRC, XChat, KVirc, Konversation, or weechat.</para>
<para>Note: if you turn this on and notifications stop appearing on
your channel, you need to turn off IRC's color filter on that channel.
To do this you will need op privileges; issue the command "/mode
&lt;channel&gt; -c" with &lt;channel&gt; replaced by your channel name.
You may need to first issue the command "/msg chanserv set
&lt;channel&gt; MLOCK +nt-slk".</para>
</listitem>
</varlistentry>
<varlistentry>
<term>maxchannels</term>
<listitem>
<para>Interpreted as an integer. If not zero, limits the number of
channels the hook will interpret from the "channels" variable.</para>
<para>This variable cannot be set through VCS configuration variables
or <filename>irker.conf</filename>; it can only be set with a command-line
argument. Thus, on a forge site in which repository owners are not
allowed to modify their post-commit scripts, a site administrator can set it
to prevent shotgun spamming by malicious project owners. Setting it to
a value less than 2, however, would probably be unwise.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>cialike</term>
<listitem>
<para>If not empty and not "None" (the default), this emulates the old
CIA behavior of dropping long lists of files in favor of a summary of
the form (N files in M directories). The value must be numeric giving
a threshold value for the length of the file list in
characters.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>template</term>
<listitem>
<para>Set the template used to generate notification messages. Only
available in VCses with config variables; presently this means git or
hg. All basic commit and extractor fields, including color switches,
are available as %() substitutions.</para>
</listitem>
</varlistentry>
</variablelist>
<para>irkerhook.py will run under both python 2 and python 3, but it does
not support mercurial repositories under python 3 yet.</para>
<refsect2 id="git"><title>git</title>
<para>Under git, the normal way to invoke this hook (from within the
update hook) passes it a refname followed by a list of commits. Because
<command>git rev-list</command> normally lists from most recent to oldest,
you'll want to use --reverse to make notifications be omitted in chronological
order. In a normal update script, the invocation should look like this</para>
<programlisting>
refname=$1
old=$2
new=$3
irkerhook.py --refname=${refname} $(git rev-list --reverse ${old}..${new})
</programlisting>
<para>except that you'll need an absolute path for irkerhook.py.</para>
<para>For testing purposes and backward compatibility, if you invoke
<application>irkerhook.py</application> with no arguments (as in a
post-commit hook) it will behave as though it had been called like
this:</para>
<programlisting>
irkerhook.py --refname=refs/heads/master HEAD
</programlisting>
<para>However, this will not give the right result when you push to
a non-default branch of a bare repo.</para>
<para>A typical way to install this hook is actually in the
<filename>post-receive</filename> hook, because it gets all the
necessary details and will not abort the push on failure. Use the
following script:</para>
<programlisting>
#!/bin/sh
echo "sending IRC notification"
while read old new refname; do
irkerhook --refname=${refname} $(git rev-list --reverse ${old}..${new})
done
</programlisting>
<para>Preferences may be set in the repo <filename>config</filename>
file in an [irker] section. Here is an example of what that can look
like:</para>
<programlisting>
[irker]
project = gpsd
color = ANSI
channels = irc://chat.freenode.net/gpsd,irc://chat.freenode.net/commits
</programlisting>
<para> You should not set the "repository" variable (an equivalent
will be computed). No attempt is made to interpret an
<filename>irker.conf</filename> file.</para>
<para>The default value of the "project" variable is the basename
of the repository directory. The default value of the "urlprefix"
variable is "cgit".</para>
<para>There is one git-specific variable, "revformat", controlling
the format of the commit identifier in a notification. It
may have the following values:</para>
<variablelist>
<varlistentry>
<term>raw</term>
<listitem><para>full hex ID of commit</para></listitem>
</varlistentry>
<varlistentry>
<term>short</term>
<listitem><para>first 12 chars of hex ID</para></listitem>
</varlistentry>
<varlistentry>
<term>describe</term>
<listitem><para>describe relative to last tag, falling back to short</para></listitem>
</varlistentry>
</variablelist>
<para>The default is 'describe'.</para>
</refsect2>
<refsect2 id="svn"><title>Subversion</title>
<para>Under Subversion, <application>irkerhook.py</application>
accepts a --repository option with value (the absolute pathname of the
Subversion repository) and a commit argument (the numeric revision level of
the commit). The defaults are the current working directory and HEAD,
respectively.</para>
<para>Note, however, that you <emphasis>cannot</emphasis> default the
repository argument inside a Subversion post-commit hook; this is
because of a limitation of Subversion, which is that getting the
current directory is not reliable inside these hooks. Instead, the
values must be the two arguments that Subversion passes to that hook
as arguments. Thus, a typical invocation in the post-commit script
will look like this:</para>
<programlisting>
REPO=$1
REV=$2
irkerhook.py --repository=$REPO $REV
</programlisting>
<para>Other --variable=value settings may also be
given on the command line, and will override any settings in an
<filename>irker.conf</filename> file.</para>
<para>The default for the project variable is the basename of the
repository. The default value of the "urlprefix" variable is
"viewcvs".</para>
<para>If an <filename>irker.conf</filename> file exists in the repository
root directory (not the checkout directory but where internals such as the
"format" file live) the hook will interpret variable settings from it. Here
is an example of what such a file might look like:</para>
<programlisting>
# irkerhook variable settings for the irker project
project = irker
channels = irc://chat.freenode/irker,irc://chat.freenode/commits
tcp = false
</programlisting>
<para>Don't set the "repository" or "commit" variables in this file;
that would have unhappy results.</para>
<para>There are no Subversion-specific variables.</para>
</refsect2>
<refsect2 id="hg"><title>Mercurial</title>
<para>Under Mercurial, <application>irkerhook.py</application> can be
invoked in two ways: either as a Python hook (preferred) or as a
script.</para>
<para>To call it as a Python hook, add the collowing to the
"commit" or "incoming" hook declaration in your Mercurial
repository:</para>
<programlisting>
[hooks]
incoming.irker = python:/path/to/irkerhook.py:hg_hook
</programlisting>
<para>When called as a script, the hook accepts a --repository option
with value (the absolute pathname of the Mercurial repository) and can
take a commit argument (the Mercurial hash ID of the commit or a
reference to it). The default for the repository argument is the
current directory. The default commit argument is '-1', designating
the current tip commit.</para>
<para>As for git, in both cases all variables may be set in the repo
<filename>hgrc</filename> file in an [irker] section. Command-line
variable=value arguments are accepted but not required for script
invocation. No attempt is made to interpret an
<filename>irker.conf</filename> file.</para>
<para>The default value of the "project" variable is the basename
of the repository directory. The default value of the "urlprefix"
variable is the value of the "web.baseurl" config value, if it
exists.</para>
</refsect2>
<refsect2 id="filter"><title>Filtering</title>
<para>It is possible to filter commits before sending them to
<application>irkerd</application>.</para>
<para>You have to specify the <option>filtercmd</option> option, which
will be the command <application>irkerhook.py</application> will
run. This command should accept one arguments, which is a JSON
representation of commit and extractor metadata (including the
channels variable). The command should emit to standard output a JSON
representation of (possibly altered) metadata.</para>
<para>Below is an example filter:</para>
<programlisting>
#!/usr/bin/env python3
# This is a trivial example of a metadata filter.
# All it does is change the name of the commit's author.
#
import sys, json
metadata = json.loads(sys.argv[1])
metadata['author'] = "The Great and Powerful Oz"
print json.dumps(metadata)
# end
</programlisting>
<para>Standard error is available to the hook for progress and
error messages.</para>
</refsect2>
</refsect1>
<refsect1 id='options'><title>OPTIONS</title>
<para><application>irkerhook.py</application> takes the following
options:</para>
<variablelist>
<varlistentry>
<term>-n</term>
<listitem><para>Suppress transmission to a daemon. Instead, dump the
generated JSON request to standard output. Useful for
debugging.</para></listitem>
</varlistentry>
<varlistentry>
<term>-V</term>
<listitem><para>Write the program version to stdout and
terminate.</para></listitem>
</varlistentry>
</variablelist>
</refsect1>
<refsect1 id='see_also'><title>SEE ALSO</title>
<para>
<citerefentry><refentrytitle>irkerd</refentrytitle><manvolnum>8</manvolnum></citerefentry>,
</para>
</refsect1>
<refsect1 id='authors'><title>AUTHOR</title>
<para>Eric S. Raymond <email>esr@snark.thyrsus.com</email>. See the
project page at <ulink
url='http://www.catb.org/~esr/irker'>http://www.catb.org/~esr/irker</ulink>
for updates and other resources.</para>
</refsect1>
</refentry>

20
org.catb.irkerd.plist Normal file
View file

@ -0,0 +1,20 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>KeepAlive</key>
<true/>
<key>Label</key>
<string>org.catb.irkerd</string>
<key>ProgramArguments</key>
<array>
<string>/usr/bin/irkerd</string>
</array>
<key>RunAtLoad</key>
<true/>
<key>UserName</key>
<string>nobody</string>
<key>GroupName</key>
<string>nobody</string>
</dict>
</plist>

1
requirements.txt Normal file
View file

@ -0,0 +1 @@
PySocks==1.5.6

268
security.adoc Normal file
View file

@ -0,0 +1,268 @@
= Security analysis of irker =
This is an analysis of security and DoS vulnerabilities associated
with irker, exploring and explaining certain design choices. Much of
it derives from a code audit and report by Daniel Franke.
== Assumptions and Goals ==
We begin by stating some assumptions about how irker will be deployed,
and articulating a set of security goals.
Communication flow in an irker deployment will look like this:
-----------------------------------------------------------------------------
Committers
|
|
Version-control repositories
|
|
irkerhook.py
|
|
irkerd
|
|
IRC servers
-----------------------------------------------------------------------------
Here are our assumptions:
1. The repositories are hosted on a public forge sites such as
SourceForge, GitHub, Gitorious, Savannah, or Gna and must be
accessible to untrusted users.
2. Repository project owners can set properties on their repositories
(including but not limited to irker.*), and may be able to set custom
post-commit hooks which can execute arbitrary code on the repository
server. In particular, these people my be able to modify the local
copy of irkerhook.py.
3. The machine which hosts irkerd has the same owner as the machine which
hosts the the repo; these machines are possibly but not necessarily
one and the same.
4. The network is protected by a perimeter firewall, and only a
trusted group is able to emit arbitrary packets from inside the
perimeter; committers are not necessarily part of this group.
5. irkerd communicates with IRC servers over the open internet,
and an IRC server's administrator is assumed to hold no position of
trust with any other party.
We can, accordingly, identify the following groups of security
principals:
A. irker administrators.
B. Project committers.
C. Project owners
D. IRC server administrators.
E. Other people on irker's internal network.
F. irkerd-IRC men-in-the-middle (i.e. people who control the network path
between irkerd and the IRC server).
G. Random people on the internet.
Our security goals for irker can be enumerated as follows:
* Control: We don't want anyone outside group A gaining control of
the machines which host irkerd or the git repos.
* Availability: Only group A should be able to to deny or degrade
irkerd's ability to receive commit messages and relay them to the
IRC server. We recognize and accept as inevitable that MITMs (groups
E and F) can do this too (by ARP spoofing, cable-cutting, etc.).
But, in particular, we would like irker-mediated services to be
resilient against DoS (denial of service) attacks.
* Authentication/integrity: Notifications should be truthful, i.e.,
commit messages sent to IRC channels should actually reflect that a
corresponding commit has taken place. We accept that groups A, C,
D, and E can violate this property.
* Secrecy: irker shouldn't aid spammers (group G) in harvesting
committers' email addresses.
* Auditability: If people abuse irkerd, we want to be able to identify
the abusive account or IP address.
== Control Issues ==
We have audited the irker and irkerhook.py code for exploitable
vulnerabilities. We have not found any in the code itself, and the
use of Python gives us confidence in the absence of large classes of errors
(such as buffer overruns) that afflict C programs.
However, the fact that irkerhook.py relies on external binaries to
mine data out of its repository opens up a well-known set of
vulnerabilities if a malicious user is able to insert binaries in a
carelessly-set execution path. Normal precautions against this should
be taken.
== Availability ==
=== Solved problems ===
When the original implementation of irkerd saw a nick collision it
generated new nicks in a predictable sequence. A malicious IRC user
could have continuously changed his own nick to the next one that
irkerd is going to try. Some randomness has been added to nick
generation to prevent this.
=== Unsolved problems ===
DoS attacks on any networked application can never completely
prevented, only mitigated by forcing attackers to invest more
resources. Here we consider the easiest attack paths against irker,
and possible countermeasures.
irker handles each connection to a particular IRC server in a separate
thread - actually, due to server limits on open channels per
connection, there may be multiple sessions per server. This may not
scale well, especially on 32-bit architectures.
Thread instance overhead, combined with the lack of any restriction on
how many URLs can appear in the 'to' list, is a DoS vulnerability. If
a repository's properties specify that notifications should go to more
than about 500 unique hostnames, then on 32-bit architectures we'll
hit the 4GB cap on virtual memory (even while the resident set size
remains small).
Another ceiling to watch out for is the ulimit on file descriptors,
which defaults to 1024 on many Linux systems but can safely be set
much larger. Each connection instance costs a file descriptor.
We consider some possible ways of addressing the problem:
1. Limit the number of URLs in a request. Pretty painless - it will
be very rare that anyone wants to specify a larger set than a project
channel plus freenode #commits - but also ineffective. A malicious
hook could achieve DoS simply by spamming lots of requests.
2. Limit the total number of requests than can be queued. Completely
ineffective - just sets a target for the DoS attack.
3. Limit the number of requests that can be queued by source IP address.
This might be worth doing; it would stymie a single-source DoS attack through
a publicly-exposed irkerd, though not a DDoS by a botnet. But there isn't
a lot of win here for a properly installed irker (e.g. behind a firewall),
which is typically going to get all its requests from a single repo host
anyway.
4. Rate-limit requests by source IP address - that is, after any request
discard additional ones during some timeout period. Again, good for
stopping a single-source DoS against an exposed irker, won't stop a
DDoS. The real problem though, is that any such rate limit might interfere
with legitimate high-volume use by a very active repo site.
After this we appear to have run out of easy options, as source IP address
is the only thing irkerd can see that an attacker can't spoof.
We mitigate some availability risks by reaping old sessions when we're
near resource limits. An ordinary DoS attack would then be prevented
from completely blocking all message traffic; the cost would be a
whole lot of join/leave spam due to connection churn.
== Authentication/Integrity ==
One way to help prevent DoS attacks would be in-band authentication -
requiring irkerd submitters to present a credential along with each
message submission. In principle this, if it existed, could also be used
to verify that a submitter is authorized to issue notifications with
respect to a given project.
We rejected this approach. The design goal for irker was to make
submissions fast, cheap, and stateless; baking an authentication
system directly into the irkerd codebase would have conflicted with
these objectives, not to mention probably becoming the camel's nose
for a godawful amount of code bloat.
The deployment advice in the installation instructions assumes that
irkerd submitters are "authenticated" by being inside a firewall - that is,
mesages are issued from an intranet and it can be trusted that anyone
issuing messages from within a given intranet is authorized to do so.
This fits the assumption that irker instances will run on forge sites
receiving requests from instances of irkerhook.py.
One larger issue (not unique to irker) is that because of the
insecured nature of IRC it is essentially impossible to secure
#commits against commit notifications that are either garbled by
software errors and misconfigurations or maliciously crafted to
confuse anyone attempting to gather statistics from that channel. The
lesson here is that IRC monitoring isn't a good method for that
purpose; going direct to the repositories via a toolkit such as Ohloh
is a far better idea.
When this analysis was originally written, we recommended using spiped
or stunnel to solve the problem of passing notifications from irkerd
to IRC servers over a potentially hostile network that might interfere
with them. Later, SSL/TLS support proved easy to add and is now in
irkerd itself.
== Secrecy ==
irkerd has no inherent secrecy risks.
The distributed version of irkerhook.py removes the host part of
author addresses specifically in order to prevent address harvesting
from the notifications.
== Auditability ==
We previously noted that source IP address is the only thing irker can
see that an attacker can't spoof. This makes auditability difficult
unless we impose conventions on the notifications passing though it.
The irkerhook.py that we ship inherits an auditability property from
the CIA service it was designed to replace: the first field of every
notification (terminated by a colon) is the name of the issuing
project. The only other competitor to replace CIA known to us
(kgb_bot) shares this property.
In the general case we cannot guarantee this property against
groups A and F.
== Risks relative to centralized services ==
irker and irkerhook.py were written as a replacement for the
now-defunct CIA notification service. The author has written
a critique of that service: "CIA and the perils of overengineering"
at <http://esr.ibiblio.org/?p=4540>. It is thus worth considering how
a risk assessment of CIA compares to this one.
The principal advantages of CIA from a security point of view were (a)
it provided a single point at which spam filtering and source blocking
could be done with benefit to all projects using the service, and (b)
since it had to have a database anyway for routing messages to project
channels, the incremental overhead for an authentication feature would
have been relatively low.
As a matter of fact rather than theory CIA never fully exploited
either possibility. Anyone could create a CIA project entry with
fanout to any desired set of IRC channels. Notifications were not
authenticated, so anyone could masquerade as a member of any project.
The only check on abuse was human intervention to source-block
spammers, and this was by no means completely effective - spam shipped
via CIA was occasionally seen on on the freenode #commits channel.
The principal security disadvantage of CIA was that it meant the
entire notification system was subject to single-point failure due
to software or hosting failures on cia.vc, or to DoS attacks
against the server. While there is no evidence that the site
was ever deliberately DoSed, failures were sufficiently common
that a half-hearted DoS attack might not have been even noticed.
Despite the absence of authentication, irker instances on
properly firewalled intranets do not obviously pose additional
spamming risks beyond those incurred by the CIA service. The
overall robustness of the notification system as a whole should
be greatly improved.
== Conclusions ==
The security and DoS issues irker has are not readily addressable by
changing the irker codebase itself, short of a complete (much more
complex and heavyweight) redesign. They are largely implicit risks of
its operating environment and must be managed by properly controlling
access to irker instances.