diff options
Diffstat (limited to 'src/env_parallel.pod')
| -rw-r--r-- | src/env_parallel.pod | 935 |
1 files changed, 935 insertions, 0 deletions
diff --git a/src/env_parallel.pod b/src/env_parallel.pod new file mode 100644 index 0000000..a198258 --- /dev/null +++ b/src/env_parallel.pod @@ -0,0 +1,935 @@ +#!/usr/bin/perl -w + +# SPDX-FileCopyrightText: 2021-2022 Ole Tange, http://ole.tange.dk and Free Software and Foundation, Inc. +# SPDX-License-Identifier: GFDL-1.3-or-later +# SPDX-License-Identifier: CC-BY-SA-4.0 + +=encoding utf8 + +=head1 NAME + +env_parallel - export environment to GNU parallel + + +=head1 SYNOPSIS + +B<env_parallel> [--record-env|--session|--end-session] + [options for GNU Parallel] + + +=head1 DESCRIPTION + +B<env_parallel> is a shell function that exports the current +environment to GNU B<parallel>. + +If the shell function is not loaded, a dummy script will be run +instead that explains how to install the function. + +B<env_parallel> is 100 ms slower at startup than pure GNU +B<parallel>, and takes up to 30% longer to start a job (typically 15 ms). + +Due to the problem with environment space (see below) the recommended +usage is either: + + # Do --record-env into $PARALLEL_IGNORED_NAMES + env_parallel --session + + # Define whatever you want to use + myfunc() { myalias and functions $myvar work. $1.; } + alias myalias='echo Aliases' + myvar='and variables' + + # env_parallel will not export names in $PARALLEL_IGNORED_NAMES + env_parallel -S localhost myfunc ::: Hooray + +Or: + + # Record the "clean" environment (this only needs to be run once) + env_parallel --record-env + + # Optionally edit ~/.parallel/ignored_vars (only needed once) + + # Define whatever you want to use + myfunc() { myalias and functions $myvar work. $1.; } + alias myalias='echo Aliases' + myvar='and variables' + + # Use --env _ to only transfer the names not in the "empty" environment + env_parallel --env _ -S localhost myfunc ::: Hooray + +In B<csh> B<--session> is not supported: + + # Record the "clean" environment - this only needs to be run once + env_parallel --record-env + + # Optionally edit ~/.parallel/ignored_vars - only needed once + + # Define whatever you want to use + alias myalias 'echo Aliases $myvar \!*.' + set myvar='and variables' + + # Use --env _ to only transfer the names not in the "empty" environment + env_parallel --env _ -S localhost myalias ::: work + +=head2 Environment space + +By default B<env_parallel> will export all environment variables, +arrays, aliases, functions and shell options (see details for the +individual shells below). + +But this only works if the size of the current environment is smaller +than the maximal length of a command and smaller than half of the max +if running remotely. E.g. The max size of Bash's command is 128 KB, so +B<env_parallel> will fail if 'B<set | wc -c>' is bigger than 128 +KB. Technically the limit is in execve(1) which IPC::open3 uses. + +Bash completion functions are well-known for taking up well over 128 +KB of environment space and the primary reason for causing +B<env_parallel> to fail. + +Instead you can use B<--env> to specify which variables, arrays, +aliases and functions to export as this will only export those with +the given name. Or follow the recommended usage in shown in +DESCRIPTION. + + +=head1 OPTIONS + +Same as GNU B<parallel> in addition to these: + +=over 4 + +=item B<--end-session> + +Undo last B<--session> + + +=item B<--record-env> + +Record all names currently defined to be ignored every time running +B<env_parallel> in the future. + + +=item B<--session> + +Ignore all names currently defined. Aliases, variables, arrays, and +functions currently defined will not be transferred. + +But names defined I<after> running B<parallel --session> I<will> be +transferred. + +This is only valid in the running shell, and can be undone with +B<parallel --end-session>. + +You can run multiple B<--session> inside each other: + + env_parallel --session + var=not + # var is transferred + env_parallel -Slocalhost 'echo var is $var' ::: ignored + env_parallel --session + # var is not transferred + env_parallel -Slocalhost 'echo var is $var' ::: ignored + env_parallel --end-session + # var is transferred again + env_parallel -Slocalhost 'echo var is $var' ::: ignored + + + +=back + + +=head1 SUPPORTED SHELLS + +=head2 Ash + +=head3 Installation + +Put this in $HOME/.profile: + + . `which env_parallel.ash` + +E.g. by doing: + + echo '. `which env_parallel.ash`' >> $HOME/.profile + +=head3 Supported use + +B<--env> is supported to export only the variable, or alias with the +given name. Multiple B<--env>s can be given. + +B<--session> is supported. + +=over 8 + +=item aliases + + alias myecho='echo aliases' + env_parallel myecho ::: work + env_parallel -S server myecho ::: work + env_parallel --env myecho myecho ::: work + env_parallel --env myecho -S server myecho ::: work + + alias multiline='echo multiline + echo aliases' + env_parallel multiline ::: work + env_parallel -S server multiline ::: work + env_parallel --env multiline multiline ::: work + env_parallel --env multiline -S server multiline ::: work + +=item functions + + ash cannot list defined functions - thus is not supported. + +=item variables + + myvar=variables + env_parallel echo '$myvar' ::: work + env_parallel -S server echo '$myvar' ::: work + env_parallel --env myvar echo '$myvar' ::: work + env_parallel --env myvar -S server echo '$myvar' ::: work + +=item arrays + +Arrays are not supported by Ash. + +=back + +=head2 Bash + +=head3 Installation + +Put this in $HOME/.bashrc: + + . `which env_parallel.bash` + +E.g. by doing: + + echo '. `which env_parallel.bash`' >> $HOME/.bashrc + +=head3 Supported use + +B<--env> is supported to export only the variable, alias, function, or +array with the given name. Multiple B<--env>s can be given. + +B<--session> is supported. + +=over 8 + +=item aliases + + alias myecho='echo aliases' + env_parallel myecho ::: work + env_parallel -S server myecho ::: work + env_parallel --env myecho myecho ::: work + env_parallel --env myecho -S server myecho ::: work + + alias multiline='echo multiline + echo aliases' + env_parallel 'multiline {}; + echo but only when followed by a newline' ::: work + env_parallel -S server 'multiline {}; + echo but only when followed by a newline' ::: work + env_parallel --env multiline 'multiline {}; + echo but only when followed by a newline' ::: work + env_parallel --env multiline -S server 'multiline {}; + echo but only when followed by a newline' ::: work + +=item functions + + myfunc() { echo functions $*; } + env_parallel myfunc ::: work + env_parallel -S server myfunc ::: work + env_parallel --env myfunc myfunc ::: work + env_parallel --env myfunc -S server myfunc ::: work + +=item variables + + myvar=variables + env_parallel echo '$myvar' ::: work + env_parallel -S server echo '$myvar' ::: work + env_parallel --env myvar echo '$myvar' ::: work + env_parallel --env myvar -S server echo '$myvar' ::: work + +=item arrays + + myarray=(arrays work, too) + env_parallel -k echo '${myarray[{}]}' ::: 0 1 2 + env_parallel -k -S server echo '${myarray[{}]}' ::: 0 1 2 + env_parallel -k --env myarray echo '${myarray[{}]}' ::: 0 1 2 + env_parallel -k --env myarray -S server \ + echo '${myarray[{}]}' ::: 0 1 2 + +=back + +=head3 BUGS + +Due to a bug in Bash, aliases containing newlines must be followed by +a newline in the command. Some systems are not affected by this bug, +but will print a warning anyway. + +=head2 csh + +B<env_parallel> for B<csh> breaks B<$PARALLEL>, so do not use +B<$PARALLEL>. + +=head3 Installation + +Put this in $HOME/.cshrc: + + source `which env_parallel.csh` + +E.g. by doing: + + echo 'source `which env_parallel.csh`' >> $HOME/.cshrc + +=head3 Supported use + +B<--env> is supported to export only the variable, alias, or +array with the given name. Multiple B<--env>s can be given. + +=over 8 + +=item aliases + + alias myecho 'echo aliases' + env_parallel myecho ::: work + env_parallel -S server myecho ::: work + env_parallel --env myecho myecho ::: work + env_parallel --env myecho -S server myecho ::: work + +=item functions + +Not supported by B<csh>. + +=item variables + + set myvar=variables + env_parallel echo '$myvar' ::: work + env_parallel -S server echo '$myvar' ::: work + env_parallel --env myvar echo '$myvar' ::: work + env_parallel --env myvar -S server echo '$myvar' ::: work + +=item arrays with no special chars + + set myarray=(arrays work, too) + env_parallel -k echo \$'{myarray[{}]}' ::: 1 2 3 + env_parallel -k -S server echo \$'{myarray[{}]}' ::: 1 2 3 + env_parallel -k --env myarray echo \$'{myarray[{}]}' ::: 1 2 3 + env_parallel -k --env myarray -S server \ + echo \$'{myarray[{}]}' ::: 1 2 3 + +=back + + +=head2 Dash + +=head3 Installation + +Put this in $HOME/.profile: + + . `which env_parallel.dash` + +E.g. by doing: + + echo '. `which env_parallel.dash`' >> $HOME/.profile + +=head3 Supported use + +B<--env> is supported to export only the variable, or alias with the +given name. Multiple B<--env>s can be given. + +B<--session> is supported. + +=over 8 + +=item aliases + + alias myecho='echo aliases' + env_parallel myecho ::: work + env_parallel -S server myecho ::: work + env_parallel --env myecho myecho ::: work + env_parallel --env myecho -S server myecho ::: work + + alias multiline='echo multiline + echo aliases' + env_parallel multiline ::: work + env_parallel -S server multiline ::: work + env_parallel --env multiline multiline ::: work + env_parallel --env multiline -S server multiline ::: work + +=item functions + + dash cannot list defined functions - thus is not supported. + +=item variables + + myvar=variables + env_parallel echo '$myvar' ::: work + env_parallel -S server echo '$myvar' ::: work + env_parallel --env myvar echo '$myvar' ::: work + env_parallel --env myvar -S server echo '$myvar' ::: work + +=item arrays + + dash does not support arrays. + +=back + + +=head2 fish + +=head3 Installation + +Put this in $HOME/.config/fish/config.fish: + + source (which env_parallel.fish) + +E.g. by doing: + + echo 'source (which env_parallel.fish)' \ + >> $HOME/.config/fish/config.fish + +=head3 Supported use + +B<--env> is supported to export only the variable, alias, function, or +array with the given name. Multiple B<--env>s can be given. + +B<--session> is supported. + +=over 8 + +=item aliases + + alias myecho 'echo aliases' + env_parallel myecho ::: work + env_parallel -S server myecho ::: work + env_parallel --env myecho myecho ::: work + env_parallel --env myecho -S server myecho ::: work + +=item functions + + function myfunc + echo functions $argv + end + env_parallel myfunc ::: work + env_parallel -S server myfunc ::: work + env_parallel --env myfunc myfunc ::: work + env_parallel --env myfunc -S server myfunc ::: work + +=item variables + + set myvar variables + env_parallel echo '$myvar' ::: work + env_parallel -S server echo '$myvar' ::: work + env_parallel --env myvar echo '$myvar' ::: work + env_parallel --env myvar -S server echo '$myvar' ::: work + +=item arrays + + set myarray arrays work, too + env_parallel -k echo '$myarray[{}]' ::: 1 2 3 + env_parallel -k -S server echo '$myarray[{}]' ::: 1 2 3 + env_parallel -k --env myarray echo '$myarray[{}]' ::: 1 2 3 + env_parallel -k --env myarray -S server \ + echo '$myarray[{}]' ::: 1 2 3 + +=back + + +=head2 ksh + +=head3 Installation + +Put this in $HOME/.kshrc: + + source `which env_parallel.ksh` + +E.g. by doing: + + echo 'source `which env_parallel.ksh`' >> $HOME/.kshrc + +=head3 Supported use + +B<--env> is supported to export only the variable, alias, function, or +array with the given name. Multiple B<--env>s can be given. + +B<--session> is supported. + +=over 8 + +=item aliases + + alias myecho='echo aliases' + env_parallel myecho ::: work + env_parallel -S server myecho ::: work + env_parallel --env myecho myecho ::: work + env_parallel --env myecho -S server myecho ::: work + + alias multiline='echo multiline + echo aliases' + env_parallel multiline ::: work + env_parallel -S server multiline ::: work + env_parallel --env multiline multiline ::: work + env_parallel --env multiline -S server multiline ::: work + +=item functions + + myfunc() { echo functions $*; } + env_parallel myfunc ::: work + env_parallel -S server myfunc ::: work + env_parallel --env myfunc myfunc ::: work + env_parallel --env myfunc -S server myfunc ::: work + +=item variables + + myvar=variables + env_parallel echo '$myvar' ::: work + env_parallel -S server echo '$myvar' ::: work + env_parallel --env myvar echo '$myvar' ::: work + env_parallel --env myvar -S server echo '$myvar' ::: work + +=item arrays + + myarray=(arrays work, too) + env_parallel -k echo '${myarray[{}]}' ::: 0 1 2 + env_parallel -k -S server echo '${myarray[{}]}' ::: 0 1 2 + env_parallel -k --env myarray echo '${myarray[{}]}' ::: 0 1 2 + env_parallel -k --env myarray -S server \ + echo '${myarray[{}]}' ::: 0 1 2 + +=back + + +=head2 mksh + +=head3 Installation + +Put this in $HOME/.mkshrc: + + source `which env_parallel.mksh` + +E.g. by doing: + + echo 'source `which env_parallel.mksh`' >> $HOME/.mkshrc + +=head3 Supported use + +B<--env> is supported to export only the variable, alias, function, or +array with the given name. Multiple B<--env>s can be given. + +B<--session> is supported. + +=over 8 + +=item aliases + + alias myecho='echo aliases' + env_parallel myecho ::: work + env_parallel -S server myecho ::: work + env_parallel --env myecho myecho ::: work + env_parallel --env myecho -S server myecho ::: work + + alias multiline='echo multiline + echo aliases' + env_parallel multiline ::: work + env_parallel -S server multiline ::: work + env_parallel --env multiline multiline ::: work + env_parallel --env multiline -S server multiline ::: work + +=item functions + + myfunc() { echo functions $*; } + env_parallel myfunc ::: work + env_parallel -S server myfunc ::: work + env_parallel --env myfunc myfunc ::: work + env_parallel --env myfunc -S server myfunc ::: work + +=item variables + + myvar=variables + env_parallel echo '$myvar' ::: work + env_parallel -S server echo '$myvar' ::: work + env_parallel --env myvar echo '$myvar' ::: work + env_parallel --env myvar -S server echo '$myvar' ::: work + +=item arrays + + myarray=(arrays work, too) + env_parallel -k echo '${myarray[{}]}' ::: 0 1 2 + env_parallel -k -S server echo '${myarray[{}]}' ::: 0 1 2 + env_parallel -k --env myarray echo '${myarray[{}]}' ::: 0 1 2 + env_parallel -k --env myarray -S server \ + echo '${myarray[{}]}' ::: 0 1 2 + +=back + + +=head2 pdksh + +=head3 Installation + +Put this in $HOME/.profile: + + source `which env_parallel.pdksh` + +E.g. by doing: + + echo 'source `which env_parallel.pdksh`' >> $HOME/.profile + +=head3 Supported use + +B<--env> is supported to export only the variable, alias, function, or +array with the given name. Multiple B<--env>s can be given. + +B<--session> is supported. + +=over 8 + +=item aliases + + alias myecho="echo aliases"; + env_parallel myecho ::: work; + env_parallel -S server myecho ::: work; + env_parallel --env myecho myecho ::: work; + env_parallel --env myecho -S server myecho ::: work + +=item functions + + myfunc() { echo functions $*; }; + env_parallel myfunc ::: work; + env_parallel -S server myfunc ::: work; + env_parallel --env myfunc myfunc ::: work; + env_parallel --env myfunc -S server myfunc ::: work + +=item variables + + myvar=variables; + env_parallel echo "\$myvar" ::: work; + env_parallel -S server echo "\$myvar" ::: work; + env_parallel --env myvar echo "\$myvar" ::: work; + env_parallel --env myvar -S server echo "\$myvar" ::: work + +=item arrays + + myarray=(arrays work, too); + env_parallel -k echo "\${myarray[{}]}" ::: 0 1 2; + env_parallel -k -S server echo "\${myarray[{}]}" ::: 0 1 2; + env_parallel -k --env myarray echo "\${myarray[{}]}" ::: 0 1 2; + env_parallel -k --env myarray -S server \ + echo "\${myarray[{}]}" ::: 0 1 2 + +=back + + +=head2 sh + +=head3 Installation + +Put this in $HOME/.profile: + + . `which env_parallel.sh` + +E.g. by doing: + + echo '. `which env_parallel.sh`' >> $HOME/.profile + +=head3 Supported use + +B<--env> is supported to export only the variable, or alias with the +given name. Multiple B<--env>s can be given. + +B<--session> is supported. + +=over 8 + +=item aliases + + sh does not support aliases. + +=item functions + + myfunc() { echo functions $*; } + env_parallel myfunc ::: work + env_parallel -S server myfunc ::: work + env_parallel --env myfunc myfunc ::: work + env_parallel --env myfunc -S server myfunc ::: work + +=item variables + + myvar=variables + env_parallel echo '$myvar' ::: work + env_parallel -S server echo '$myvar' ::: work + env_parallel --env myvar echo '$myvar' ::: work + env_parallel --env myvar -S server echo '$myvar' ::: work + +=item arrays + + sh does not support arrays. + +=back + + +=head2 tcsh + +B<env_parallel> for B<tcsh> breaks B<$PARALLEL>, so do not use +B<$PARALLEL>. + +=head3 Installation + +Put this in $HOME/.tcshrc: + + source `which env_parallel.tcsh` + +E.g. by doing: + + echo 'source `which env_parallel.tcsh`' >> $HOME/.tcshrc + +=head3 Supported use + +B<--env> is supported to export only the variable, alias, or +array with the given name. Multiple B<--env>s can be given. + +=over 8 + +=item aliases + + alias myecho 'echo aliases' + env_parallel myecho ::: work + env_parallel -S server myecho ::: work + env_parallel --env myecho myecho ::: work + env_parallel --env myecho -S server myecho ::: work + +=item functions + +Not supported by B<tcsh>. + +=item variables + + set myvar=variables + env_parallel echo '$myvar' ::: work + env_parallel -S server echo '$myvar' ::: work + env_parallel --env myvar echo '$myvar' ::: work + env_parallel --env myvar -S server echo '$myvar' ::: work + +=item arrays with no special chars + + set myarray=(arrays work, too) + env_parallel -k echo \$'{myarray[{}]}' ::: 1 2 3 + env_parallel -k -S server echo \$'{myarray[{}]}' ::: 1 2 3 + env_parallel -k --env myarray echo \$'{myarray[{}]}' ::: 1 2 3 + env_parallel -k --env myarray -S server \ + echo \$'{myarray[{}]}' ::: 1 2 3 + +=back + + +=head2 Zsh + +=head3 Installation + +Put this in $HOME/.zshrc: + + . `which env_parallel.zsh` + +E.g. by doing: + + echo '. `which env_parallel.zsh`' >> $HOME/.zshenv + +=head3 Supported use + +B<--env> is supported to export only the variable, alias, function, or +array with the given name. Multiple B<--env>s can be given. + +B<--session> is supported. + +=over 8 + +=item aliases + + alias myecho='echo aliases' + env_parallel myecho ::: work + env_parallel -S server myecho ::: work + env_parallel --env myecho myecho ::: work + env_parallel --env myecho -S server myecho ::: work + + alias multiline='echo multiline + echo aliases' + env_parallel multiline ::: work + env_parallel -S server multiline ::: work + env_parallel --env multiline multiline ::: work + env_parallel --env multiline -S server multiline ::: work + +=item functions + + myfunc() { echo functions $*; } + env_parallel myfunc ::: work + env_parallel -S server myfunc ::: work + env_parallel --env myfunc myfunc ::: work + env_parallel --env myfunc -S server myfunc ::: work + +=item variables + + myvar=variables + env_parallel echo '$myvar' ::: work + env_parallel -S server echo '$myvar' ::: work + env_parallel --env myvar echo '$myvar' ::: work + env_parallel --env myvar -S server echo '$myvar' ::: work + +=item arrays + + myarray=(arrays work, too) + env_parallel -k echo '${myarray[{}]}' ::: 1 2 3 + env_parallel -k -S server echo '${myarray[{}]}' ::: 1 2 3 + env_parallel -k --env myarray echo '${myarray[{}]}' ::: 1 2 3 + env_parallel -k --env myarray -S server \ + echo '${myarray[{}]}' ::: 1 2 3 + +=back + + +=head1 EXIT STATUS + +Same as GNU B<parallel>. + + +=head1 AUTHOR + +When using GNU B<env_parallel> for a publication please cite: + +O. Tange (2018): GNU Parallel 2018, March 2018, ISBN 9781387509881, +DOI: 10.5281/zenodo.1146014. + +This helps funding further development; and it won't cost you a cent. +If you pay 10000 EUR you should feel free to use GNU Parallel without citing. + +Copyright (C) 2007-10-18 Ole Tange, http://ole.tange.dk + +Copyright (C) 2008-2010 Ole Tange, http://ole.tange.dk + +Copyright (C) 2010-2022 Ole Tange, http://ole.tange.dk and Free +Software Foundation, Inc. + + +=head1 LICENSE + +This program is free software; you can redistribute it and/or modify +it under the terms of the GNU General Public License as published by +the Free Software Foundation; either version 3 of the License, or +at your option any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU General Public License for more details. + +You should have received a copy of the GNU General Public License +along with this program. If not, see <http://www.gnu.org/licenses/>. + +=head2 Documentation license I + +Permission is granted to copy, distribute and/or modify this +documentation under the terms of the GNU Free Documentation License, +Version 1.3 or any later version published by the Free Software +Foundation; with no Invariant Sections, with no Front-Cover Texts, and +with no Back-Cover Texts. A copy of the license is included in the +file LICENSES/GFDL-1.3-or-later.txt. + + +=head2 Documentation license II + +You are free: + +=over 9 + +=item B<to Share> + +to copy, distribute and transmit the work + +=item B<to Remix> + +to adapt the work + +=back + +Under the following conditions: + +=over 9 + +=item B<Attribution> + +You must attribute the work in the manner specified by the author or +licensor (but not in any way that suggests that they endorse you or +your use of the work). + +=item B<Share Alike> + +If you alter, transform, or build upon this work, you may distribute +the resulting work only under the same, similar or a compatible +license. + +=back + +With the understanding that: + +=over 9 + +=item B<Waiver> + +Any of the above conditions can be waived if you get permission from +the copyright holder. + +=item B<Public Domain> + +Where the work or any of its elements is in the public domain under +applicable law, that status is in no way affected by the license. + +=item B<Other Rights> + +In no way are any of the following rights affected by the license: + +=over 2 + +=item * + +Your fair dealing or fair use rights, or other applicable +copyright exceptions and limitations; + +=item * + +The author's moral rights; + +=item * + +Rights other persons may have either in the work itself or in +how the work is used, such as publicity or privacy rights. + +=back + +=back + +=over 9 + +=item B<Notice> + +For any reuse or distribution, you must make clear to others the +license terms of this work. + +=back + +A copy of the full license is included in the file as +LICENCES/CC-BY-SA-4.0.txt + + +=head1 DEPENDENCIES + +B<env_parallel> uses GNU B<parallel>. + + +=head1 SEE ALSO + +B<parallel>(1), B<ash>(1), B<bash>(1), B<csh>(1), B<dash>(1), +B<fish>(1), B<ksh>(1), B<pdksh>(1) B<tcsh>(1), B<zsh>(1). + + +=cut |