diff options
Diffstat (limited to 'src/sem.pod')
-rw-r--r-- | src/sem.pod | 381 |
1 files changed, 381 insertions, 0 deletions
diff --git a/src/sem.pod b/src/sem.pod new file mode 100644 index 0000000..15d4ea7 --- /dev/null +++ b/src/sem.pod @@ -0,0 +1,381 @@ +#!/usr/bin/perl -w + +# SPDX-FileCopyrightText: 2021-2022 Ole Tange, http://ole.tange.dk and Free Software and Foundation, Inc. +# SPDX-License-Identifier: GFDL-1.3-or-later +# SPDX-License-Identifier: CC-BY-SA-4.0 + +=head1 NAME + +sem - semaphore for executing shell command lines in parallel + +=head1 SYNOPSIS + +B<sem> [--fg] [--id <id>] [--semaphoretimeout <secs>] [-j <num>] [--wait] command + +=head1 DESCRIPTION + +GNU B<sem> is an alias for GNU B<parallel --semaphore>. + +GNU B<sem> acts as a counting semaphore. When GNU B<sem> is called +with command it starts the command in the background. When I<num> +number of commands are running in the background, GNU B<sem> waits for +one of these to complete before starting the command. + +GNU B<sem> does not read any arguments to build the command (no -a, +:::, and ::::). It simply waits for a semaphore to become available +and then runs the command given. + +Before looking at the options you may want to check out the examples +after the list of options. That will give you an idea of what GNU +B<sem> is capable of. + +=head1 OPTIONS + +=over 9 + +=item I<command> + +Command to execute. The command may be followed by arguments for the +command. + + +=item B<--bg> + +Run command in background thus GNU B<sem> will not wait for +completion of the command before exiting. This is the default. + +In toilet analogy: GNU B<sem> waits for a toilet to be available, +gives the toilet to a person, and exits immediately. + +See also: B<--fg> + + +=item B<--jobs> I<N> + +=item B<-j> I<N> + +=item B<--max-procs> I<N> + +=item B<-P> I<N> + +Run up to N commands in parallel. Default is 1 thus acting like a +mutex. + +In toilet analogy: B<-j> is the number of toilets. + + +=item B<--jobs> I<+N> + +=item B<-j> I<+N> + +=item B<--max-procs> I<+N> + +=item B<-P> I<+N> + +Add N to the number of CPU cores. Run up to this many jobs in +parallel. For compute intensive jobs B<-j> +0 is useful as it will run +number-of-cpu-cores jobs simultaneously. + + +=item B<--jobs> I<-N> + +=item B<-j> I<-N> + +=item B<--max-procs> I<-N> + +=item B<-P> I<-N> + +Subtract N from the number of CPU cores. Run up to this many jobs in +parallel. If the evaluated number is less than 1 then 1 will be used. +See also B<--use-cpus-instead-of-cores>. + + +=item B<--jobs> I<N>% + +=item B<-j> I<N>% + +=item B<--max-procs> I<N>% + +=item B<-P> I<N>% + +Multiply N% with the number of CPU cores. Run up to this many jobs in +parallel. If the evaluated number is less than 1 then 1 will be used. +See also B<--use-cpus-instead-of-cores>. + + +=item B<--jobs> I<procfile> + +=item B<-j> I<procfile> + +=item B<--max-procs> I<procfile> + +=item B<-P> I<procfile> + +Read parameter from file. Use the content of I<procfile> as parameter +for I<-j>. E.g. I<procfile> could contain the string 100% or +2 or +10. + + +=item B<--pipe> + +Pass stdin (standard input) to I<command>. + +If I<command> read from stdin (standard input), use B<--pipe>. + + +=item B<--semaphorename> I<name> + +=item B<--id> I<name> + +Use B<name> as the name of the semaphore. Default is the name of the +controlling tty (output from B<tty>). + +The default normally works as expected when used interactively, but +when used in a script I<name> should be set. I<$$> or I<my_task_name> +are often a good value. + +The semaphore is stored in ~/.parallel/semaphores/ + +In toilet analogy the name corresponds to different types of toilets: +e.g. male, female, customer, staff. + + +=item B<--fg> + +Do not put command in background. + +In toilet analogy: GNU B<sem> waits for a toilet to be available, +takes a person to the toilet, waits for the person to finish, and +exits. + + +=item B<--semaphoretimeout> I<secs> + +=item B<--st> I<secs> + +If I<secs> > 0: If the semaphore is not released within I<secs> +seconds, take it anyway. + +If I<secs> < 0: If the semaphore is not released within I<secs> +seconds, exit. + +In toilet analogy: I<secs> > 0: If no toilet becomes available within +I<secs> seconds, pee on the floor. I<secs> < 0: If no toilet becomes +available within I<secs> seconds, exit without doing anything. + + +=item B<--wait> + +Wait for all commands to complete. + +In toilet analogy: Wait until all toilets are empty, then exit. + + +=back + +=head1 UNDERSTANDING A SEMAPHORE + +Try the following example: + + sem -j 2 'sleep 1;echo 1 finished'; echo sem 1 exited + sem -j 2 'sleep 2;echo 2 finished'; echo sem 2 exited + sem -j 2 'sleep 3;echo 3 finished'; echo sem 3 exited + sem -j 2 'sleep 4;echo 4 finished'; echo sem 4 exited + sem --wait; echo sem --wait done + +In toilet analogy this uses 2 toilets (B<-j 2>). GNU B<sem> takes '1' +to a toilet, and exits immediately. While '1' is sleeping, another GNU +B<sem> takes '2' to a toilet, and exits immediately. + +While '1' and '2' are sleeping, another GNU B<sem> waits for a free +toilet. When '1' finishes, a toilet becomes available, and this GNU +B<sem> stops waiting, and takes '3' to a toilet, and exits +immediately. + +While '2' and '3' are sleeping, another GNU B<sem> waits for a free +toilet. When '2' finishes, a toilet becomes available, and this GNU +B<sem> stops waiting, and takes '4' to a toilet, and exits +immediately. + +Finally another GNU B<sem> waits for all toilets to become free. + + +=head1 EXAMPLE: Gzipping *.log + +Run one gzip process per CPU core. Block until a CPU core becomes +available. + + for i in *.log ; do + echo $i + sem -j+0 gzip $i ";" echo done + done + sem --wait + +=head1 EXAMPLE: Protecting pod2html from itself + +pod2html creates two files: pod2htmd.tmp and pod2htmi.tmp which it +does not clean up. It uses these two files for a short time. But if +you run multiple pod2html in parallel (e.g. in a Makefile with make +-j) there is a risk that two different instances of pod2html will +write to the files at the same time: + + # This may fail due to shared pod2htmd.tmp/pod2htmi.tmp files + foo.html: + pod2html foo.pod --outfile foo.html + + bar.html: + pod2html bar.pod --outfile bar.html + + $ make -j foo.html bar.html + +You need to protect pod2html from running twice at the same time. +B<sem> running as a mutex will make sure only one runs: + + foo.html: + sem --id pod2html pod2html foo.pod --outfile foo.html + + bar.html: + sem --id pod2html pod2html bar.pod --outfile bar.html + + clean: foo.html bar.html + sem --id pod2html --wait + rm -f pod2htmd.tmp pod2htmi.tmp + + $ make -j foo.html bar.html clean + +=head1 BUGS + +None known. + + +=head1 REPORTING BUGS + +Report bugs to <bug-parallel@gnu.org>. + + +=head1 AUTHOR + +Copyright (C) 2010-2022 Ole Tange, http://ole.tange.dk and Free +Software Foundation, Inc. + + +=head1 LICENSE + +This program is free software; you can redistribute it and/or modify +it under the terms of the GNU General Public License as published by +the Free Software Foundation; either version 3 of the License, or +at your option any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU General Public License for more details. + +You should have received a copy of the GNU General Public License +along with this program. If not, see <https://www.gnu.org/licenses/>. + +=head2 Documentation license I + +Permission is granted to copy, distribute and/or modify this +documentation under the terms of the GNU Free Documentation License, +Version 1.3 or any later version published by the Free Software +Foundation; with no Invariant Sections, with no Front-Cover Texts, and +with no Back-Cover Texts. A copy of the license is included in the +file LICENSES/GFDL-1.3-or-later.txt. + +=head2 Documentation license II + +You are free: + +=over 9 + +=item B<to Share> + +to copy, distribute and transmit the work + +=item B<to Remix> + +to adapt the work + +=back + +Under the following conditions: + +=over 9 + +=item B<Attribution> + +You must attribute the work in the manner specified by the author or +licensor (but not in any way that suggests that they endorse you or +your use of the work). + +=item B<Share Alike> + +If you alter, transform, or build upon this work, you may distribute +the resulting work only under the same, similar or a compatible +license. + +=back + +With the understanding that: + +=over 9 + +=item B<Waiver> + +Any of the above conditions can be waived if you get permission from +the copyright holder. + +=item B<Public Domain> + +Where the work or any of its elements is in the public domain under +applicable law, that status is in no way affected by the license. + +=item B<Other Rights> + +In no way are any of the following rights affected by the license: + +=over 2 + +=item * + +Your fair dealing or fair use rights, or other applicable +copyright exceptions and limitations; + +=item * + +The author's moral rights; + +=item * + +Rights other persons may have either in the work itself or in +how the work is used, such as publicity or privacy rights. + +=back + +=back + +=over 9 + +=item B<Notice> + +For any reuse or distribution, you must make clear to others the +license terms of this work. + +=back + +A copy of the full license is included in the file as +LICENCES/CC-BY-SA-4.0.txt + + +=head1 DEPENDENCIES + +GNU B<sem> uses Perl, and the Perl modules Getopt::Long, +Symbol, Fcntl. + + +=head1 SEE ALSO + +B<parallel>(1) + +=cut |