.\" This file is part of rex - a remote execution utility -*- nroff -*-
.\" Copyright (C) 2012-2022 Sergey Poznyakoff
.\"
.\" Rex is free software; you can redistribute it and/or modify
.\" it under the terms of the GNU General Public License as published by
.\" the Free Software Foundation; either version 3, or (at your option)
.\" any later version.
.\"
.\" Rex is distributed in the hope that it will be useful,
.\" but WITHOUT ANY WARRANTY; without even the implied warranty of
.\" MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
.\" GNU General Public License for more details.
.\"
.\" You should have received a copy of the GNU General Public License
.\" along with Rex. If not, see .
.TH REX 8 "March 17, 2022" "REX" "Rex User Reference"
.ds ET /etc
.SH NAME
rex \- remote execution utility
.SH SYNOPSIS
\fBrex\fR\
[\fB\-NVdlnvh\fR]\
[\fB\-c\fR \fIFILE\fR]\
[\fB\-\-config=\fIFILE\fR]\
[\fB\-\-noop\fR]\
[\fB\-\-no\-init\-file\fR]\
[\fB\-\-no\-rc\-file\fR]\
[\fB\-\-no\-resolve\fR]\
[\fB\-\-verbose\fR]\
[\fB\-\-version\fR]\
[\fB\-\-help\fR]\
\fICOMMAND\fR\
[\fIARG\fR...]
.SH DESCRIPTION
.B Rex
runs a supplied command or shell script on several hosts in
succession or in parallel. It can also be used to copy a file or
files to several hosts.
.PP
The two main \fICOMMAND\fRs are \fBrun\fR, which runs the specified
command on multiple hosts, and \fBcopy\fR, or \fBcp\fR, which copies
files to them.
.PP
For both commands, there are two ways to select hosts to operate upon.
.PP
The first way is the \fB\-H\fR (\fB\-\-host\fR) option. Its argument
is either a single hostname or IP, or a comma-separated list of
hostnames or IP addresses. The option adds these hosts to the list.
Multiple options can be specified.
.PP
The \fB\-H\fR option exists mainly as a way of quickly running a job
of several hosts at once. If you have a set of hosts which you often
work with, it's best to use a
.IR hostgroup .
.PP
Hostgroup defines a list of hosts and, optionally, a set of
.B rex
settings to use when running commands on them. Each hostgroup is
identified by its name and is defined in a separate disk file. See
the subsection \fBHostgroups\fR, for a detailed discussion of how to
define a hostgroup.
.PP
Once defined, the option \fB\-g\fR (\fB\-\-group\fR) can be used to
run a command on each host in the group. The argument to the option
is the name of the group. Thus, e.g.
.PP
.EX
rex run \-g\fINAME\fR \fICOMMAND\fR
.EE
.PP
runs \fICOMMAND\fR on hosts from hostgroup \fINAME\fR, and
.PP
.EX
rex copy \-g\fINAME\fR \fIFILE\fR /tmp
.EE
.PP
copies \fIFILE\fR to the directory \fB/tmp\fR on each host from this
hostgroup.
.PP
If the \fB\-g\fR option is omitted, \fBrex\fR will use the hostgroup
from the top of the hostgroup stack or, if it is empty, the
default hostgroup.
.PP
\fIHostgroup stack\fR is maintained using the \fBrex group\fR command.
Once the user pushes a hostgroup name on it, that hostgroup will be
used by any subsequent \fBrex\fR command, unless it is given the
\fB\-g\fR option. The \fBrex group\fR command provides ways to pop
items off the stack, swap arbitrary element and the top of the stack,
select new hostgroups on the fly, etc. See the section
.B REX GROUP
below for a detailed discussion of hostgroup stack.
.PP
The \fBdefault hostgroup\fR is used if the hostgroup stack is empty.
It is defined using the \fBoption group\fR statement in the
configuration file.
.PP
The list of hosts in a hostgroup can be modified for the duration of a
single \fBrex\fR command by using the \fB\-H\fR and \fB\-X\fR options.
The \fB\-H\fR (\fB\-\-host\fR) option works exactly as described
above. The \fB\-i\fR (\fB\-\-ignore\-hosts\fR) option can be given
to ignore the list of hosts defined in the hostgroup and operate only
on those supplied with the \fB\-\-host\fR options.
.PP
The \fB\-X\fR (\fB\-\-exclude\fR) option instructs \fBrex\fR to remove
its argument from the list of hosts. Its argument is the domain name
or IP address of the host. It need not be exactly the same name
or IP as the one used in the \fBset hosts\fR statement in the \fBrc\fR
file. It suffices that it resolves to the same IP as one of the hosts
listed in the hostgroup.
.PP
The list of all available hostgroups can be obtained using the
\fBlist\fR command. The list is organized in two columns:
group name and its short description (if available).
.SS Login credentials
Normally, remote hosts should be configured to accept logins to the
user account without password, by using shared keys. However, it is
also possible to work on hosts requiring password for authentication.
To do so, \fBrex\fR keeps the database of login credentials. The
database is edited using the command
.PP
.EX
rex edit
.EE
.PP
This command starts the editor defined by the \fBVISUAL\fR (or, if
unset, \fBEDITOR\fR) environment variable. Upon exit from the editor,
it compares the new content with the existing one. If changed, the
user is prompted whether they wish to save the changes.
.PP
The database is a dictionary of \fIkey-value\fR pairs. A key
identifies the credential (e.g. \fBuser\fR, for username, or
\fBpass\fR, for password). Unless prefixed with one or more
\fIqualifiers\fR, it is valid for all servers in all hostgroups.
Qualifiers limit the scope of a key to a given hostgroup or server.
.PP
In general, the syntax for a key is:
.PP
.EX
[[\fIGROUP\fB:\fR][\fIHOST\fR]\fB:\fR]\fIKEYWORD\fR
.EE
.PP
When looking for a key, the most qualified match is preferred, so the
lookup order is:
.RS
.nr step 1 1
.IP \n[step].
\fIGROUP\fB:\fIHOST\fB:\fIKEYWORD\fR
.IP \n+[step].
\fIHOST\fB:\fIKEYWORD\fR
.IP \n+[step].
\fIGROUP\fB::\fIKEYWORD\fR
.IP \n+[step].
\fIKEYWORD\fR
.RE
.PP
For example, to determine the login name for the given host, \fBrex\fR
looks up for the keyword \fBuser\fR. If no matching key is found
and the the \fB\-u\fR (\fB\-\-user\fR) option is given, its value is
used. Otherwise, the name of the value of the environment variable
.B USER
is used.
.PP
Similarly, the password to use is determined by the following
look-ups:
.RS
.nr step 1 1
.IP \n[step].
\fIGROUP\fB:\fIHOST\fB:\fBpass\fR
.IP \n+[step].
\fIHOST\fB:pass\fR
.IP \n+[step].
\fIGROUP\fB::\fBpass\fR
.IP \n+[step].
\fBpass\fR
.RE
.PP
If none of these keys is found, the value of the \fB\-p\fR
(\fB\-\-password\fR) option is used. If this option is not supplied
either, passwordless account is assumed.
.SS Configuration files
Upon startup \fBrex\fR looks for the following files and attempts to
source them:
.TP
.B \*(ET/rex/rc
System-wide configuration file.
.TP
.B ~/.rex/rc
User-specific configuration file.
.PP
These files are not required to exist. If they do the user settings
override the system-wide ones.
.PP
These files are sourced as regular \fBTCL\fR code. Normally, they
modify the \fBrex\fR settings by using a st of commands provided for
that purpose:
.TP
\fBhost\fR \fIHOSTNAME\fR [\fIHOSTNAME\fR...]
Adds the listed hostnames to the list of hosts.
.TP
\fBhostname\fR \fIIP\fR \fINAME\fR
Registers \fINAME\fR in DNS cache as a hostname for the IPv4
address \fIIP\fR. This is useful if \fIIP\fR has no PTR record, or its
PTR record is deemed unsuitable for some reason.
.TP
\fBifmode\fR \fB{\fR \fImode\fR \fIbody\fR [\fImode\fR \fIbody\fR...] \fB}\fR
Matches the current mode against each of \fImode\fR arguments. As soon
as the match is found, the corresponding \fBbody\fR is evaluated.
Similarly to the
.BR switch (n)
command, a dash in place of \fIbody\fR means that the body for the
next mode should also be used as the body for this one. If the next
mode has a body of \fB\(dq\-\(dq\fR, than the body after that is used,
etc.) The \fImode\fR should be one of the following:
.BR run ,
.BR copy\-to ,
.BR copy\-from ,
.BR login ,
.BR list ,
and
.BR edit .
If the last mode used is the word \fBdefault\fR, it will match any
mode not explicitly listed before.
The \fBifmode\fR pattern allows for selecting different settings
depending on the running mode. For example, it is common to set the
\fBTERM\fR environment variable to \fBdumb\fR for \fBrun\fR mode.
However, it is undesirable in \fBlogin\fR and \fBedit\fR modes, and
has no effect on the remaining modes. To make sure it is set only for
\fBrun\fR mode, the following statement is normally used:
.RS
.EX
ifmode {
run { environ TERM=dump }
}
.EE
.RE
.TP
\fBoption\fR \fIOPTNAME\fR \fIVALUE\fR [\fIVALUE\fR...]
Sets or unsets a \fBrex\fR option. \fIOPTNAME\fR is the long name of
the option without the leading two dashes. For example, setting
.RS
.EX
option jobs 25
.EE
has the same effect as giving the option \fB\-\-jobs=25\fR in the
command line.
Options that take no argument in the command line are treated as
booleans. The \fIVALUE\fR of
.BR on ,
.BR true ,
.BR yes ,
or
.BR 1
sets the option, while the \fBVALUE\fR of
.BR off ,
.BR false ,
.BR no ,
or
.BR 0
unsets it. Thus, to set the \fBno\-resolve\fR option, use
.EX
option no\-resolve true
.EE
.RE
.TP
\fBtimeout\fR [\fINUMBER\fR]
Sets the timeout for expecting reply from the remote commands.
\fINUMBER\fR is the timeout value in seconds. When used without
arguments, returns the current timeout value.
.TP
\fBenviron\fR [\fB\-set\fR] \fINAME\fB=\fIVALUE\fR [\fINAME\fB=\fIVALUE\fR...]
Sets environment variables.
.TP
\fBenviron -unset\fR \fINAME\fR [\fINAME\fR...]
Unsets environment variables.
.TP
\fBearlycmd\fR [\fB\-clear\fR] \fICOMMAND\fR [\fICOMMAND\fR...]
Defines the sequence of shell commands that will be sent as soon as
the secure connection to the remote host is established, before
waiting for the command line prompt. For an example of its use,
see the note about \fBZSH\fR, below.
.TP
\fBshrc\fR [\fB\-clear\fR] \fICOMMAND\fR [\fICOMMAND\fR...]
Defines the sequence of shell commands that will be sent after
receiving the first command line prompt from the remote server.
.SS Hostgroups
A \fIhostgroup\fR defines a list of hosts and optionally associates
a set of
.B rex
settings to use when running commands on them. Each hostgroup is
identified by its name, which must comply with the same requirements
as a name of a disk file. The hostgroup definition consists of one or
more files located in the directory \fB~/.rex/hostgroup/\fINAME\fR
(user-specific), or \fB\*(ET/rex/hostgroup/\fINAME\fR (system-wide), where
\fINAME\fR is the name of the group. User-specific directories are
given preference over the system-wide ones. The only mandatory file is
\fBrc\fR. It is a \fBTCL\fR source, which will be loaded in a
separate namespace, to avoid the possibility of inadvertent
contamination of the main \fBrex\fR code.
.PP
The hostgroup \fBrc\fR file should use the \fBhost\fR and/or
\fBhostgroup\fR commands
(see the subsection
.B Configuration files
above) to define the list of hosts in the hostgroup.
.SS Debugging
To see what's going on during \fBrex\fR invocation, use the
\fB\-d (\fB\-\-debug\fR) option. This option is incremental, that is
the more times you repeat it in the command line, the more debugging
information is output. Currently, the most detailed information is
obtained at level 3, i.e. using \fB\-ddd\fR.
.PP
Additional information is provided by the \fB\-l\fR
(\fB\-\-log\fR) option, which enables logging of the send/expect
dialog to stdout.
.PP
If you wish to get even more information and are familiar with
.BR expect (1)
internals, set the
.B EXPECT_DEBUG
environment variable prior to invoking \fBrex\fR. The value of this
variable is the name of a file where to output additional information
(as well as showing it on the standard error). Overall effect is
equivalent to setting \fBexp_internal\fR in
.BR expect .
.PP
When \fBrex\fR encounters an error in one of its rc files, it prints
the location where the error occurred and a short diagnostic message
describing the nature of the error. If you wish to obtain full trace
of TCL calls for such errors, use the
.B \-\-tcl\-trace
option. Normally, this option is only useful for \fBrex\fR developers.
.SH OPTIONS
.TP
\fB\-c\fR, \fB\-\-config=\fIFILE\fR
Read configuration from \fIFILE\fR.
.TP
.BR \-d ", " \-\-debug
Increase debugging level.
.TP
.BR \-l ", " \-\-log
Log everything to the standard output.
.TP
\fB\-q\fR, \fB\-\-no\-init\-file\fR, \fB\-\-no\-rc\-file\fR
Don't read system-wide and user configuration files.
.TP
.BR \-N ", " \-\-noop
Ignore all commands (useful for side effects).
.TP
.BR \-n ", " \-\-no\-resolve
Don't resolve IP addresses when printing hostnames.
.TP
.B \-\-tcl\-trace
Produce full TCL trace if an error occurs while sourcing one of the
configuration files. This option is useful mainly for developers.
.TP
\fB\-v\fR, \fB\-\-verbose\fR
Enable additional warnings.
.TP
.BR \-V ", " \-\-version
Print program version and copyright statement.
.TP
.BR \-h ", " \-\-help
Show a terse help summary
.SH REX RUN
.EX
\fBrex run \fIPROGRAM\fR \fIARG\fR [\fIARG\fR...]
.EE
.PP
The \fBrex run\fR command runs the specified program with the
arguments on the servers.
.PP
.SS Sequential and parallel execution
By default, \fBrex run\fR executes the command on each host in
succession. The output from the command is printed on the standard
output, preceded by a header line containing the name of the remote host
that produced the output, and a colon.
.PP
The \fB\-\-no\-host\-header\fR option disables printing of that header
line. The \fB\-P\fR (\fB\-\-prefix\fR) option further modifies the
output by prefixing each line with the hostname, a greater than sign,
and a space character.
.PP
When operating on a big number of hosts, the operation can be sped up
by running command on several hosts at once. This parallel operation
is enabled by the \fB\-j\fIN\fR (\fB\-\-jobs=\fIN\fR) option. The
\fIN\fR parameter is the number of hosts to be processed in a batch.
.PP
The \fB\-j\fR option implies \fB\-\-prefix \-\-no\-host\-header\fR
to make it clear which host produced which line. However, if the
command produces multiple lines of output, they become intermixed when
running in parallel. To further improve readability of the output,
the \fB\-b\fR (\fB\-\-buffer\-output\fR) option is provided.
.PP
This option instructs \fBrex\fR to buffer output from each host in the
batch and to output it only after the host closes connection. This
way, output from each host appears on screen as a contiguous text.
.PP
The \fB\-b\fR option cancels the implied \fB\-\-prefix
\-\-no\-host\-header\fR settings.
.SS Transferring data prior to execution
If the program relies on some additional files to be present, these can
be transferred to each host prior to running the program. The
transfer is requested by the \fB\-D\fR (\fB\-\-data\-file\fR) option.
The argument to this option names the file to be copied to the target host.
.PP
To transfer multiple files, repeat the \fB\-D\fR option for each of
them.
.PP
The files supplied with the \fB\-D\fR options will be copied to a
temporary directory created in \fB/tmp\fR on each target host. That
directory will become current working directory when invoking the
\fIPROGRAM\fR.
.PP
The arguments to \fIPROGRAM\fR can refer to the files transferred via
the \fB\-D\fR option using the syntax \fB{\fIN\fB}\fR, where \fIN\fR
is the number of the \fB\-D\fR option occurrence (1-based). The
\fB{\fIN\fB}\fR is expanded to the file name part of the argument to
the corresponding \fB\-D\fR option. For example,
.PP
.EX
rex run \-D prefix.txt \-D repl sed \-f {2} {1} /etc/passwd
.EE
.PP
will result in running the following command in a temporary directory
on each target host:
.PP
.EX
sed \-f repl prefix.txt /etc/passwd
.EE
.PP
Note, that the \fB{\fIN\fB}\fR notation expands to a file name relative to
the current working directory on the remote host. This means that
.PP
.EX
rex run \-D /tmp/prefix.txt \-D repl sed \-f {2} {1} /etc/passwd
.EE
.PP
will result in exactly the same command as the example above: the
expansion of \fB{1}\fR will throw away the directory part from the
file name.
.PP
Similarly, the token \fB{}\fR expands to a space-separated
list of file names passed with all \fB\-D\fR options, in the same
order they appear on the command line.
.PP
The \fIPROGRAM\fR itself can be an interpreted script that needs to be
copied on each host for execution. The \fB\-\-copy\fR option
instructs \fBrex\fR to create a temporary directory on the remote
host, copy \fIPROGRAM\fR there, and then select the interpreter to run
by analyzing the first line from it. If this line starts with
characters \fB#!\fR, the rest of line is taken as the interpreter
pathname and initial arguments. For example, assuming that file
\fB/libexec/foo\fR is a shell script beginning with \fB#!/bin/ksh\fR,
the following command:
.PP
.EX
rex run \-\-copy /libexec/foo /home
.EE
.PP
will first copy the file \fB/libexec/foo\fR to a temporary
directory on the remote host, and then invoke it as
.PP
.EE
/bin/ksh foo /home
.EX
.PP
If the file does not begin with \fB#!\fR, the default interpreter,
\fB/bin/sh\fR, will be used. The default interpreter can be changed
using the \fB\-\-interpreter\fR option, as in the following example:
.PP
.EX
rex run \-\-copy \-\-interpreter="/bin/env perl" test.pl
.EE
.SS Scripting
When given the \fB\-s \fINAME\fR option, \fBrex run\fR looks for the file
named \fINAME\fB\.tcl\fR in the library search path. The search path
is scanned from left to right. The first encountered
\fINAME\fB\.tcl\fR file is sourced in the namespace \fBhostproc\fR.
.PP
The default library search path is:
.PP
.RS
.B ~/.rex/script
.br
.B \*(ET
.RE
.PP
If the \fB\-g \fIGROUP\fR (\fB\-\-group=\fIGROUP\fR) option is given,
the path is modified as follows:
.PP
.RS
\fB~/.rex/\fIGROUP\fB/script\fR
.br
\fB\*(ET/\fIGROUP\fB/script\fR
.br
.B ~/.rex/script
.br
.B \*(ET
.RE
.PP
The following functions from the \fBhostproc\fR namespace are used by
\fBrex\fR:
.TP
.B proc prologue {}
Called at the very beginning of \fBrex\fR invocation, after analyzing
the command line options, but before parsing the remaining command
line arguments.
This function can be used to initialize internal variables. It is
also allowed to modify the global \fBconfig\fR array (see the section
\fBCONFIG ARRAY\fR, below).
In the following example, \fBprologue\fR is used to set up the command
line:
.sp
.RS
.EX
proc prologue {} {
global config
set config(argv) {ls \-ld /home}
}
.EE
.RE
.TP
.B proc epilogue {}
Called after processing all hosts.
It an be used for application-specific cleanup.
.TP
.B proc transition {host state}
Called when the session for \fI$host\fR is switched to the state
\fI$state\fR. Currently implemented states are:
.RS
.TP
.B INIT
The initial state. It is entered after starting \fBssh\fR for that host.
.TP
.B PASS_SENT
Entered after receiving password prompt from the remote side and
responding to it with a password.
.TP
.B SUDO
Entered after responding to the password prompt received in the
\fBCOMMAND\fR state. This happens only in sudo mode, when \fBsudo\fR
asks for credentials.
.TP
.B SUDO2
Entered after sending a \fBsudo rm\fR command to remove the temporary
directory.
.TP
.B STTY
Entered after sending the initial \fBstty \-echo\fR command, which
happens after successful authentication. The session remains in this
state while sending commands from the initial command list (see the
description of \fBshrc\fR key in \fBCONFIG ARRAY\fR.
.TP
.B COMMAND
Entered after sending the main command to the host.
.TP
.B STATUS
Entered when the command prompt is first seen after issuing the
command. It is used by the \fBrex group select\fR to collect the
return status of the command.
.TP
.B LOGOUT
The very last state in the session lifetime. It is entered after
sending the terminating \fBexit\fR command to the host.
The following implementation of the \fBtransition\fR function uses
this information to display informational message about each host
being processed:
.sp
.RS
.EX
proc transition {host state} {
if {$state == "INIT"} {
puts \-nonewline stderr "$host... "
} elseif {$state == "LOGOUT"} {
puts "done"
}
}
.EE
.RE
.RE
.TP
.B proc getline {host line}
Invoked each time a new line of output is received from the host. The
\fBhost\fR parameter is the name of the host. The \fB$line\fR
parameter is the line received from the host (with trailing \fBCRLF\fR
characters).
.TP
.B proc finish {host refname status}
The \fBfinish\fR function is called when EOF has been received from
the \fBhost\fR. The \fBrefname\fR parameter is a name of the variable
keeping the entire text received from that host during the session.
This variable exists only if \fBrex\fR has been instructed to buffer
host output (see the \fB\-b\fR option). If it exists, the output text
is stored as a list of lines. The \fBstatus\fR parameter is the exit
status of the command.
.SS REX RUN OPTIONS
.TP
\fB\-b\fR, \fB\-\-buffer\-output\fR
Output from servers is buffered and printed only when the job is
finished. When running several jobs in parallel (see \fB\-\-jobs\fR
option below), output lines from the concurrent jobs are printed as
they arrive and can therefore appear on the screen intermixed. This
option ensures that output from each server is printed as a contiguous
block of text instead.
.TP
.B \-\-copy
copy \fIPROGRAM\fR to each host before executing
.TP
\fB\-D\fR, \fB\-\-data\-file=\fIFILE\fR
Declares \fIFILE\fR as a data file which will be used by
\fICOMMAND\fR. The file will be copied to each host prior to running
the command on it.
Multiple files can be specified by repeating the \fB\-D\fR option as
many times as needed.
If \fB\-D \fIFILE\fR option is given, the string \fB{}\fR passed as one of the
\fIARGS\fR is replaced by \fIFILE\fR on the target machine. If
several \fB\-D\fR options are given, \fB{}\fR is replaced by their
arguments. For example, the command
.B rex run -D a -D b proc {}
copies files \fBa\fR and \fBb\fR to the target machine and then runs
the command \fBproc a b\fR.
.TP
\fB\-g\fR, \fB\-\-group=\fINAME\fR
Select host group.
.TP
\fB\-h\fR, \fB\-\-help\fR
Display short help summary.
.TP
\fB\-H\fR, \fB\-\-host=\fIHOST[\fB,\fINAME\fR...]
Add \fIHOST\fR to the list. Multiple hosts can be specified as a
comma-delimited list (no whitespace around commas, please).
.TP
\fB\-i\fR, \fB\-\-ignore\-hosts\fR
Ignore the list of hosts read from the hostgroup file. Operate only
on hosts supplied in the command line (see the \fB\-\-host\fR option,
above). This is useful to preserve the settings from the hostgroup
file while running the command on the hosts defined on the command line.
.TP
.BR \-I ", " \-\-interactive
Interactively ask for missing usernames and passwords.
.TP
\fB\-\-interpreter=\fICOMMAND\fR
Use \fICOMMAND\fR as interpreter for running \fIPROGRAM\fR. This
option implies
.BR \-\-copy .
By default, \fBrex\fR deduces the interpreter to use from
the first line of \fIPROGRAM\fR.
.TP
\fB\-j\fR, \fB\-\-jobs=\fIN\fR
Run \fIN\fR jobs at once. Implies
.BR "\-\-no\-host\-header \-\-prefix" .
.TP
.B \-\-no\-host\-header
Don't print hostname before output from the host.
.TP
\fB\-p\fR, \fB\-\-password=\fIPWD\fR
Set default password (unsafe!)
.TP
\fB\-P\fR, \fB\-\-prefix\fR
Prefix each output line with the host name or IP.
.TP
\fB\-s\fR, \fB\-\-source=\fINAME\fR
Source .rex/\fINAME\fR.tcl after reading configuration files.
.TP
.B \-\-sudo
Run the command via
.BR sudo (1).
.TP
\fB\-u\fR, \fB\-\-user=\fINAME\fR
Set default user name.
.TP
.BR \-w ", " \-\-confirm
Prompt and wait for confirmation before each host.
.TP
\fB\-X\fR, \fB\-\-exclude\-host=\fIHOST[\fB,\fINAME\fR...]
Remove \fIHOST\fR from the list. Multiple hosts can be specified as a
comma-delimited list (no whitespace around commas, please).
.TP
\fB\-Z\fR, \fB\-\-zsh\-quirk\fR
Try to cope with hosts running
.BR zsh (1).
Unless special measures are taken, the output produced by \fBzsh\fR
is heavily contaminated with special escape sequences, which makes it
almost impossible to recognize command line prompts.
The \fB\-Z\fR option forces \fBrex\fR to set \fBTERM=dumb\fR. Then,
if no command line prompt is seen within the predefined interval,
.B rex
will try to disable \fBZLE\fR option, set \fBPS1\fR to
a dollar sign followed by space and will retry scanning. This allows for
properly recognizing command line prompts returned by \fBzsh\fR at the
expense of a certain slowdown. See the subsection \fBZSH\fR below for
more efficient ways of handling \fBzsh\fR.
.SH REX COPY
.EX
\fBrex copy \fR[\fIOPTIONS\fR] \fIFILE\fR [\fIFILE\fR...] \fIDEST\fR
.EE
.PP
The \fBrex copy\fR command copies one or more \fIFILE\fRs to the given
destination (\fIDEST\fR) on each remote host.
.PP
The command verb can be abbreviated to \fBcp\fR.
.SS REX COPY OPTIONS
.TP
\fB\-g\fR, \fB\-\-group=\fINAME\fR
Select host group.
.TP
\fB\-h\fR, \fB\-\-help\fR
Display short help summary.
.TP
\fB\-H\fR, \fB\-\-host=\fIHOST[\fB,\fINAME\fR...]
Add \fIHOST\fR to the list. Multiple hosts can be specified as a
comma-delimited list (no whitespace around commas, please).
.TP
\fB\-i\fR, \fB\-\-ignore-hosts\fR
Ignore the list of hosts read from the hostgroup file. Operate only
on hosts supplied in the command line (see the \fB\-\-host\fR option,
above). This is useful to preserve the settings from the hostgroup
file while executing the command on the hosts defined on the command line.
.TP
.BR \-I ", " \-\-interactive
Interactively ask for missing usernames and passwords.
.TP
\fB\-j\fR, \fB\-\-jobs=\fIN\fR
Run \fIN\fR jobs at once. Implies
.BR "\-\-no\-host\-header \-\-prefix" .
.TP
\fB\-p\fR, \fB\-\-password=\fIPWD\fR
Set default password (unsafe!)
.TP
\fB\-P\fR, \fB\-\-prefix\fR
Prefix each output line with the host name or IP.
.TP
.B \-\-sudo
Switch to root privileges before copying to \fIDEST\fR. Technically,
speaking, the command
\fBrex copy \-\-sudo \fIFILE DEST\fR
is equivalent to
\fBrex run \-D \fIFILE\fB sudo cp {} \fIDEST\fR
.TP
\fB\-u\fR, \fB\-\-user=\fINAME\fR
Set default user name.
.TP
.BR \-w ", " \-\-confirm
Prompt and wait for confirmation before proceeding with each host.
.TP
\fB\-X\fR, \fB\-\-exclude\-host=\fIHOST[\fB,\fINAME\fR...]
Remove \fIHOST\fR from the list. Multiple hosts can be specified as a
comma-delimited list (no whitespace around commas, please).
.SH REX COPY\-FROM
.EX
\fBrex copy\-from \fR[\fIOPTIONS\fR] \fIHOST\fR \fIFILE\fR [\fIFILE\fR...] \fIDEST\fR
.EE
.PP
The \fBcopy\-from\fR command copies the requested files from
\fIHOST\fR to the given destination \fIDEST\fR on the local machine.
Basically, it is equivalent to running \fBscp\fR with proper
arguments, except that it is more convenient to use if \fIHOST\fR
doesn't allow for passwordless authentication. In that case \fBrex
copy\-from\fR will use the authentication credentials from the rex
database, as described in the subsection \fBLogin credentials\fR,
above.
.PP
The command verb \fBrcp\fR (an abbreviation for \fIreverse copy\fR),
can be used instead of \fBcopy\-from\fR.
.SS REX COPY\-FROM OPTIONS
.TP
.BR \-I ", " \-\-interactive
Interactively ask for missing usernames and passwords.
.TP
\fB\-p\fR, \fB\-\-password=\fIPWD\fR
Set default password (unsafe!)
.TP
\fB\-u\fR, \fB\-\-user=\fINAME\fR
Set default user name.
.TP
\fB\-h\fR, \fB\-\-help\fR
Display short help summary.
.SH REX GROUP
The \fBrex group\fR command manages host group stack.
.TP
\fBrex group push \fIGROUP\fR
Pushes hostgroup \fIGROUP\fR on stack. The \fIGROUP\fR will be used
by all subsequent \fBrex\fR commands, in the absence of the \fB\-g\fR option.
.TP
.B rex group pop
Pops the hostgroup off the top of the stack.
.TP
\fBrex group swap \fIN\fR
Exchanges top of the stack with the \fIN\fRth element (0-based).
.TP
\fBrex group drop \fIN\fR
Removes the \fIN\fRth element from the stack.
.TP
\fBrex group select \fICOMMAND ARGS\fR...
Using the current hostgroup, selects the hosts on which the supplied
shell command gives success (exits with 0 status). On success, creates
a new unnamed hostgroup consisting of the selected hosts and pushes it
on stack.
.TP
\fBrex group show \fR[\fIOPTIONS\fR]
Shows the hostgroup stack. Without options, displays only the topmost
(current) entry. The following options are understood:
.RS
.TP
\fB\-a\fR, \fB\-\-all\fR
Show all stack entries.
.TP
\fB\-H\fR, \fB\-\-hosts\fR
Show hosts in the topmost (current) hostgroup.
.PP
All options are mutually exclusive.
.RE
.PP
All the commands above accept also the \fB\-h\fR (\fB\-\-help\fR)
option, which displays a short help summary.
.SH REX LOGIN
.EX
\fBrex login [\fIOPTIONS\fR] \fIHOST\fR
.EE
.PP
Logs in to the remote host \fIHOST\fR. It is basically equivalent to
using \fBssh\fR, except that it is convenient to use if the \fIHOST\fR
doesn't support passwordless authentication (see the section
.BR "Login credentials" ,
above).
.SS REX LOGIN OPTIONS
.TP
.BR \-I ", " \-\-interactive
Interactively ask for missing usernames and passwords.
.TP
\fB\-p\fR, \fB\-\-password=\fIPWD\fR
Set default password (unsafe!)
.TP
\fB\-u\fR, \fB\-\-user=\fINAME\fR
Set default user name.
.TP
\fB\-Z\fR, \fB\-\-zsh\-quirk\fR
Try to cope with hosts running
.BR zsh (1).
See the description of this option in subsection
.BR "REX RUN OPTIONS" .
.TP
\fB\-h\fR, \fB\-\-help\fR
Display short help summary.
.SH REX LIST
.EX
\fBrex list \fR[\fBgroups\fR]
.sp
\fBrex list \fIOPTIONS\fR hosts\fR
.EE
.PP
List hostgroups or hosts in a hostgroup (second form).
.PP
The output of \fBrex list groups\fR consists of two columns: the
group name and its description.
.PP
The second form lists hosts in a hostgroup selected by \fIOPTIONS\fR.
At least one option must be present.
.SS REX LIST OPTIONS
.TP
\fB\-h\fR, \fB\-\-help\fR
Display a short help summary.
.TP
\fB\-H\fR, \fB\-\-host=\fIHOST[\fB,\fINAME\fR...]
Add \fIHOST\fR to the list. Multiple hosts can be specified as a
comma-delimited list (no whitespace around commas, please).
.TP
\fB\-i\fR, \fB\-\-ignore-hosts\fR
Ignore the list of hosts read from the hostgroup file. Operate only
on hosts supplied in the command line (see the \fB\-\-host\fR option,
above). This is useful to preserve the settings from the hostgroup
file while executing the command on the hosts defined on the command line.
.TP
\fB\-X\fR, \fB\-\-exclude\-host=\fIHOST[\fB,\fINAME\fR...]
Remove \fIHOST\fR from the list. Multiple hosts can be specified as a
comma-delimited list (no whitespace around commas, please).
.PP
Additionally, the \fB\-h\fR, or \fB\-\-help\fR option displays a short
help summary.
.SH REX EDIT
.EX
\fBrex edit \fR[\fIFILE\fB]\fR
.EE
.PP
Starts the editor defined by the \fBVISUAL\fR (or, if it is
unset, \fBEDITOR\fR) environment variable for editing the
.B rex
database file \fIFILE\fR (or \fB~/.rex/db\fR, if used without argument).
Upon exit from the editor, the command compares the new content with the
existing one. If changed, the user is prompted whether they wish to
save the changes.
.SH CONFIG ARRAY
The \fBconfig\fR variable is an associative array keeping the
configuration of \fBrex\fR. The \fBrc\fR scripts are free to modify
it as they see fit. The valid keys in \fBconfig\fR are:
.TP
.B data
(list) A list of file names to copy to each host prior to running
the command. See the description of the \fB\-\-data\-file\fR option.
.TP
.B debug
(integer) Debug verbosity level. Defaults to 0. It is increased by
one with each \fB\-d\fR (\fB\-\-debug\fR) option. Biggest possible
value is 3.
.TP
.B earlycmd
(list) A list of commands to send as early as the very first line of output
from the host is seen (see the note about \fBZSH\fR, below).
.TP
.B exclude_hosts
List of host names to exclude from processing. Normally set via the
\fB\-X\fR exclude option.
.TP
.B file
(string) Name of the file to read instead of the default configuration
files.
.TP
.B hosts
List of hosts to run command on.
.TP
.B mode
Operation mode. The value is one of:
.BR run ,
.BR copy-from ,
.BR copy-to ,
.BR edit ,
.BR group .
.BR list ,
and
.BR login .
.TP
.B pass
Password to use when no host or group-specific one is given (see \fBLogin
credentials\fR).
.TP
.B prompt
Regular expression for matching the command line prompt. Default is
\fB"(%|#|\\$) $"\fR .
.\" FIXME:
.\" .TP
.\" .B script
.TP
.B shrc
A list of commands to send after the initial command line prompt has been
received. See the note about \fBDisabling command history\fR for an
example of its use.
.TP
.B sudo
The string
.BR \(dqsudo \(dq,
if the command must be executed with super-user privileges (via
.BR sudo (9)).
An empty string otherwise.
.TP
.B user
User login name to use (see \fBLogin credentials\fR). Defaults to the
value of the environment variable
.BR USER .
.TP
.B option,buffer-output
(boolean) Buffer server output and print it only when the batch job is
finished. See the \fB\-q\fR (\fB\-\-buffer\-output\fR) option.
.TP
.B option,confirm
(boolean) Prompt and wait for confirmation before each host. See the
\fB\-w\fR, \fB\-\-confirm\fR option.
.TP
.B option,editdb
(boolean) Editdb mode. This option is set by the
.B rex edit
command. See the section
.BR "REX EDIT" .
.TP
.B option,encrypt
(boolean) String encrypt mode. See the \fBencrypt\fR option.
.TP
.B option,hostgroup
Name of the hostgroup as supplied by the \fB\-g\fR (\fB\-\-group\fR)
option.
.TP
.B option,ignore-hosts
(boolean) Ignore the list of hosts read from the hostgroup file.
Set by the \fB\-i\fR (\fB\-\-ignore\-hosts\fR) command line option.
.TP
.B option,interactive
(boolean) Set by the \fB\-I\fR (\fB\-\-interactive) option.
.TP
.B option,jobs
(integer) Number of hosts to process in one batch. Default is 1. See
the
\fB\-j\fR (\fB\-\-jobs\fR) option.
.TP
.B option,log
(boolean) If set to 1, log entire output from each server on standard
out. This is equivalent to setting \fBlog_user 1\fR in expect.
.TP
.B option,no-init-file
(boolean Don't read initialization files (\fBConfiguration files\fR).
See also the \fB\-q\fR (\fB\-\-no\-init\-file\fR) option.
.TP
.B option,no-host-header
(boolean) Don't print hostname before output from the host (the
\fB\-\-no\-host\-header\fR option).
.TP
.B option,noop
(boolean) Don't run \fICOMMAND\fR. This option is convenient when you
have some side effects in your rc file (such as, e.g. launching a VPN
etc.) and wish to evaluate them without actually executing a command.
.TP
.B option,prefix
(boolean) Prefix each output line with the host name or IP. See the
\fB\-P\fR (\fB\-\-prefix\fR) option.
.TP
.B option,script
(boolean) True if command is a local script (see the \fB\-\-script\fR
option).
.TP
.B option,source
Base name of the user-defined TCL script to be sourced. See the
\fB\-\-source\fR option.
.TP
.B option,resolve
(boolean) Resolve host IP addresses into the hostnames. Default is 1.
Reset via the \fB\-n\fR, \fB\-\-no\-resolve\fR command line option.
.TP
.B option,verbose
(boolean) Print additional warnings (\fB\-v\fR, \fB\-\-verbose\fR).
.TP
.B option,zsh-quirk
(boolean) Try to cope with remote
.BR zsh (1)
shells. See the description of the \fB\-Z\fR (\fB\-\-zsh\-quirk\fR)
option, and the subsection \fBZSH\fR in the \fBNOTES\fR section, below.
.SH REX DATABASE
\fBRex\fR searches for its database file in the following locations:
.TP
.B \*(ET/rex/db
Site-wide database file.
.TP
.B ~/.rex/db
Per-user database file.
.PP
Existing files are loaded in that order, so that per-user settings
override the system-wide ones. Each \fBdb\fR file is a textual file
with key-value pairs separated by any amount of whitespace. Each pair
occupies a separate line. Empty lines and comments (starting with
\fB#\fR) are ignored.
.PP
The format of the key value is:
.PP
.EX
[\fIGROUP\fB:\fR][\fIHOST\fB:\fR]\fIKEYWORD\fR
.EE
.PP
(square brackets denoting optional parts). The reserved keywords are:
.TP
.B user
User login name.
.TP
.B pass
User password.
.TP
.B earlycmd
Commands to send as early as the very first line of output
from the host is seen. The value is a whitespace-separated list of
TCL strings. Please use curly braces, instead of double quotes, to
prevent interpretation of the string value.
This setting is useful if the command line prompt cannot be easily
recognized (see the note about \fBZSH\fR, below).
.TP
.B shrc
Commands to send after the initial command line prompt has been
received. The value is a whitespace-separated list of
TCL strings. Please use curly braces, instead of double quotes, to
prevent interpretation of the string value.
See the note about \fBDisabling command history\fR for an example of
its use.
.\" FIXME: This one is no longer used:
.\" .TP
.\" .B sudo
.\" Password for obtaining superuser privileges via \fBsudo\fR.
.PP
Users can define their own keywords.
.SH NOTES
.SS Disabling command history
You might wish to disable the command history on remote hosts for
commands executed by \fBrex\fR. Use the \fBshrc\fR setting for that.
This setting supplies a list of commands \fBrex\fR will send after it
has received the initial command line prompt from the remote machine.
.PP
The exact list of commands depends on the shell that the remote uses.
.PP
The simplest way, common for \fBbash\fR, \fBksh\fR, and \fBzsh\fR is:
.PP
.EX
shrc HISTFILE=/dev/null
.EE
.PP
Similar setting for
.BR csh (1)
or
.BR tcsh (1)
is:
.PP
.EX
shrc {set histfile=/dev/null}
.EE
.PP
The drawback of this approach is that the assignment itself and all the
built-in commands sent by \fBrex\fR are still retained in history.
.PP
A more sophisticated way, suitable for both \fBbash\fR and \fBzsh\fR
shells is:
.PP
.EX
shrc {
if test \-n "$HISTFILE" && test \-f "$HISTFILE"; \\
then \\
mv $HISTFILE ${HISTFILE}~; \\
head \-n \-1 ${HISTFILE}~ > $HISTFILE; \\
fi} \\
{unset HISTFILE}
.EE
.PP
It removes the most recent line from the history file and unsets
the \fBHISTFILE\fR variable, thereby preventing the shell from
further updating it.
.PP
(The most recent line contains the command \fBstty \-echo\fR, sent by
\fBrex\fR at the beginning of each session.)
.PP
To set history cleaning on a per-host (or per-hostgroup) basis, use
the \fBshrc\fR key in the rex database file.
.SS ZSH
Interaction with
.BR zsh (1)
poses considerable problems because this shell spits out enormous
amounts of escape sequences for colorization and other purposes, which
makes it next to impossible to write an efficient regular expression for
recognizing command line prompts. To make matters even worse, it
ignores any changes made by \fBstty\fR.
.PP
An ad-hoc way of coping with hosts running \fBzsh\fR is provided by
the \fB\-Z\fR (\fB\-\-zsh\-quirk\fR) command line option. However, it
is convenient only for casual usage, because of the slowdown it imposes.
.PP
The proper way of handling \fBzsh\fR hosts is by configuring the
\fBearlycmd\fR setting. This configuration setting declares a list
of commands that \fBrex\fR is to send as early as it sees the very
first line of output from the host.
.PP
If all your hosts run \fBzsh\fR, add the following to your \fBrc\fR
file:
.PP
.EX
ifmode {
run {
environ TERM=dumb
}
}
earlycmd {unsetopt ZLE} {PS1='$ '}
.EE
.PP
This will disable the \fBzle\fR module, which is the cause of all evil
(TM), reset terminal defaults to decent values and set a proper
command line prompt.
.PP
If only some of your hosts run \fBzsh\fR, you can set the
\fBearlycmd\fR option in the \fBrex\fR database file, using the
\fBearlycmd\fR key, either for hostgroup or for individual hosts.
.PP
The value of that option is a whitespace-separated list of TCL strings.
.PP
For example, if the account on host \fBbaboo\fR runs \fBzsh\fR, run
\fBrex edit\fR and add the following line:
.PP
.EX
baboo:earlycmd {unsetopt ZLE} {PS1='$ '}
.EE
.PP
Notice that you will still need to have
.PP
.EX
environ TERM=dumb
.EE
.PP
in your \fBrc\fR file.
.SH FILES
.TP
.B \*(ET/rex/rc
System-wide configuration file.
.TP
.B ~/.rex/rc
User-specific configuration file.
.TP
.B \*(ET/rex/db
System-wide database file (see \fBLogin credentials\fR).
.TP
.B ~/.rex/db
User-specific database file.
.TP
.B \*(ET/rex/script
System-wide script directory (see the subsection \fBScripting\fR,
above).
.TP
.B ~/.rex/script
User-specific script directory.
.TP
\fB\*(ET/rex/hostgroup/\fIGROUP\fB/script\fR
System-wide hostgroup-specific script directory.
.TP
.B ~/.rex/hostgroup/\fIGROUP\fB/script\fR
Per-user hostgroup-specific script directory.
.TP
\fB\*(ET/rex/hostgroup/\fIGROUP\fB/rc\fR
System-wide definition of \fIGROUP\fR.
.TP
\fB~/.rex/hostgroup/\fIGROUP\fB/rc\fR
User-specific definition of \fIGROUP\fR.
.TP
\fB\*(ET/rex/hostgroup/\fIGROUP\fB/description\fR
System-wide description of \fIGROUP\fR.
.TP
\fB~/.rex/hostgroup/\fIGROUP\fB/description\fR
User-specific description of \fIGROUP\fR.
.SH ENVIRONMENT
.TP
.B EXPECT_DEBUG
Enables detailed diagnostic output of internal activity of commands
such as expect and interact. The value of this variable is the name
of a file where to output additional information (as well as showing
it on the standard error). E.g., setting \fBEXPECT_DEBUG=foo\fR is
similar to using \fBexp_internal -f foo 1\fR at the beginning of the
\fBexpect\fR code.
.TP
.B EXPECT_PROMPT
Regular expression for matching command line prompt. Default is
\fB"(%|#|\\$) $"\fR .
.TP
.B VISUAL
Pathname of the editor binary.
.TP
.B EDITOR
Pathname of the editor binary. It is used if \fBVISUAL\fR is not defined.
.TP
.B USER
Name of the current user. See also the
.BR \-u ", " \-\-user
option.
.SH "RETURN VALUE"
.TP
.B 0
Success.
.TP
.B 1
Command line usage error.
.TP
.B 2
Some of hosts were unreachable or refused connection.
.SH EXAMPLE
.nr step 1 1
.IP \n[step].
Check uptime on all hosts in group \fBweb\fR:
.PP
.EX
rex run -g web uptime
.EE
.IP \n+[step].
Same with single-line output for each host.
.PP
.EX
rex run \-g web \-\-prefix \-\-no\-host\-header uptime
.EE
.IP \n+[step].
Restart \fBhttpd\fR on each host (notice the use of \fB\-\-sudo\fR).
.PP
.EX
rex run \-g web --sudo /etc/init.d/httpd restart
.EE
.SH AUTHORS
Sergey Poznyakoff
.SH "BUG REPORTS"
Report bugs to .
.SH COPYRIGHT
Copyright \(co 2012\-2022 Sergey Poznyakoff
.br
.na
License GPLv3+: GNU GPL version 3 or later
.br
.ad
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
.\" Local variables:
.\" eval: (add-hook 'write-file-hooks 'time-stamp)
.\" time-stamp-start: ".TH [A-Z_][A-Z0-9_.\\-]* [0-9] \""
.\" time-stamp-format: "%:B %:d, %:y"
.\" time-stamp-end: "\""
.\" time-stamp-line-limit: 20
.\" end: