From 0917d1276103d9cc893e8ac091e9e63c5e6182f8 Mon Sep 17 00:00:00 2001 From: Sergey Poznyakoff Date: Mon, 22 Jul 2019 12:22:44 +0300 Subject: Document the module subsystem --- doc/wydawca.texi | 579 ++++++++++++++++++++++++++++++++++++++----------------- 1 file changed, 407 insertions(+), 172 deletions(-) (limited to 'doc') diff --git a/doc/wydawca.texi b/doc/wydawca.texi index ca58d1d..db06139 100644 --- a/doc/wydawca.texi +++ b/doc/wydawca.texi @@ -37,13 +37,10 @@ Copyright @copyright{} 2007, 2009, 2010, 2011, 2012, 2013, 2014, 2015, Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no -Invariant Sections, with the Front-Cover texts being ``A GNU Manual'', -and with the Back-Cover Texts as in (a) below. A copy of the license -is included in the section entitled ``GNU Free Documentation License''. +Invariant Sections, no Front-Cover, and no Back-Cover texts. -(a) The FSF's Back-Cover Text is: ``You have freedom to copy and modify -this GNU Manual, like GNU software. Copies published by the Free -Software Foundation raise funds for GNU development.'' +A copy of the license is included in the section entitled ``GNU Free +Documentation License''. @end copying @titlepage @@ -144,7 +141,7 @@ Mail Notification @chapter Introduction to Wydawca @cindex introduction Let's begin with a short synopsis. Suppose you run a developer's -site, like, e.g. @indicateurl{gnu.org}. You have two +site, such as e.g. @indicateurl{gnu.org}. You have two @dfn{distribution @acronym{URL}s}: @indicateurl{ftp.gnu.org}, which distributes stable versions of the software, and @indicateurl{alpha.gnu.org}, which distributes alpha and pre-test @@ -194,8 +191,8 @@ their distribution sites. @command{Wydawca} is such a release submission daemon. It is able to handle any number of @samp{source/destination} pairs (called -@dfn{spools}) in real time, and offers an extensible logging and mail -notification mechanism, allowing both package maintainers and site +@dfn{spools}) in real time, and offers extensible logging and +notification mechanisms, allowing both package maintainers and site administrators to be immediately notified about any occurring problems. @command{Wydawca} supports upload directive versions 1.1@footnote{ @@ -249,14 +246,14 @@ that case. upload and corresponding distribution directories. In @command{wydawca} terminology, upload directories are also called @dfn{source}, and distribution directories -- @dfn{destination} -directories. The configuration file also supplies all the information +directories. The configuration file supplies also the information necessary to access user and project databases. When started, @command{wydawca} scans each source directory and prepares a list of files found there. Then, it compacts this list by looking for @dfn{directive files} and re-arranging list members in @dfn{triplets}. A @dfn{directive file} is a special file that must be -supplied with each upload and that contains directive regarding the +supplied with each upload and that contains instructions regarding the placement of the uploaded files. A @dfn{triplet} is a standard entity, consisting of three files: a clear-signed directive file, a file to be distributed, and a detached signature of the latter. @@ -288,8 +285,8 @@ possibility to restart interrupted or otherwise broken uploads later. @cindex @acronym{PGP} After completing these preliminary stages, @command{wydawca} analyzes the directive file and extracts the project name -from it. Using this name as a key, it looks up in the @dfn{project -dictionary} a list of users authorized to make uploads for this +from it. Using this name as a key, it searches in the @dfn{project +dictionary} for a list of users authorized to make uploads for this project. This list contains user names and their corresponding public @acronym{PGP} keys. @command{Wydawca} tries to verify the directive file using each @acronym{PGP} key from this list, until a matching @@ -435,6 +432,11 @@ by using the following options: $ wydawca --spool=ftp --spool=test --source=/home/ftp/test-upload @end smallexample + Any non-optional arguments appearing in the command line, are +treated as the uploaders' UIDs. If present, only triplets uploaded by +these users will be processed. Of course, such UIDs can be combined +with the @option{--spool} and @option{--source} options. + @anchor{debug} @xopindex{debug, described} @sopindex{d, described} @@ -480,7 +482,7 @@ $ wydawca -c new.cfg --dry-run @dfn{configuration file} @file{wydawca.rc}. By default it is located in @var{$sysconfidr} (i.e., in most cases @file{/usr/local/etc}, or @file{/etc}), but an alternative location may be specified using -@option{--config} command line option (@pxref{starting, config-file}). +@option{--config-file} command line option (@pxref{starting, config-file}). If any errors are encountered in the configuration file, the program reports them on its error output and exits with a non-zero status. @@ -2252,20 +2254,196 @@ statistics (all, errors, warnings); @end deffn @node notification -@section Mail Notification -@cindex mail notification +@section Notification Mechanism +@cindex notification While running, @command{wydawca} keeps track of certain events occurring, such as, for example, broken @acronym{PGP} signatures or -file uploads attempted by unauthorized users. The utility can notify, -via email, project administrators about any of those events that -concern their projects. Additionally, the system administrator can -choose to obtain via email the execution statistics, described in the -previous section. +file uploads attempted by unauthorized users. It can issue +notifications about such events using the supplied loadable modules. + +Configuration of notifications consists of two parts. First the +required loadable module must be loaded and configured. Then, +configure the notification itself. + +@menu +* modules:: +* event notification:: +* mod_mailutils:: Mail Notification +* mod_logstat:: Notification to syslog +@end menu + +@node modules +@subsection modules + +A @dfn{loadable module} is a piece of software that provides +notification mechanism for @command{wydawca}. It is built as a UNIX +dynamically loaded library and placed in one of the preconfigured +directories which constitute a @dfn{library load path}. To load a +module, the following statement is used: + +@deffn {Config} module @var{name} @var{file} +Load the module @var{name} from @var{file}. Other places of the +configuration file can refer to the module as @var{name}. + +The @var{file} argument is a file name of the module (normally, a +@samp{file.so} or @samp{file.la} file). +@end deffn + +Unless @var{file} in the @samp{module} statement it is an absolute +file name, it will be searched in the library load path, which is +defined as: + +@enumerate 1 +@item +Optional @dfn{prefix} search directories specified by the +@samp{module-prepend-load-path} directive (see below). + +@item +@command{Wydawca} module directory: @samp{@var{$prefix}/lib/wydawca}. + +@item +Additional search directories specified by the @command{module-load-path} +directive (see below). + +@item +The value of the environment variable @env{LTDL_LIBRARY_PATH}. + +@item +The system dependent library search path (e.g. on GNU/Linux it is +defined by the file @file{/etc/ld.so.conf} and the environment variable +@env{LD_LIBRARY_PATH}). +@end enumerate + +The value of @env{LTDL_LIBRARY_PATH} and @env{LD_LIBRARY_PATH} must be a +colon-separated list of absolute directory names, for example +@samp{/usr/lib/mypkg:/lib/foo}. + +In any of these directories, @command{wydawca} first attempts to find +and load the given filename. If this fails, it tries to append the following +suffixes to it: + +@enumerate 1 +@item the libtool archive suffix: @samp{.la} + +@item the suffix used for native dynamic libraries on the host platform, +e.g., @samp{.so}, @samp{.sl}, etc. +@end enumerate + +The statements that modify the module search path are: + +@deffn {Config} module-load-path @var{list} +This directive adds the directories listed in its argument to the +module load path. Example: + +@example +module-load-path (/usr/lib/wydawca,/usr/local/wydawca/lib); +@end example +@end deffn + +@deffn {Config} module-prepend-load-path @var{list} +Same as above, but the directories from @var{list} are added to the +beginning of the module search list, rather than to its end. The +order of directories in @var{list} is preserved in both cases. +@end deffn + +Once loaded, the module can be initialized. This is done by the +following block statement: + +@deffn {Config} module-init @var{name} @{ ... @} +Initialize the module identified by @var{name}. The module must have +been previously loaded using the @samp{module} statement, as described +above. The statements between curly braces are module-specific +configuration statements. See the module descriptions below for a +detailed discussion of these. +@end deffn + +@node event notification +@subsection Event Notification + + A number of @dfn{events} are tracked during the execution. Any of +them can be used to trigger the notification mechanism. It is +configured using the following statement: + +@deffn {Config} notify-event @{ ... @} +@smallexample +@group +notify-event @{ + # Event on which to notify + event @var{eid}; - The sender email address for these notifications is set using the -@code{from-address} statement. + # Name of the module to invoke on event + module @var{modname}; -@deffn {Config} from-address address + # Module-specific configuration data + module-config @{ + ... + @} +@} +@end group +@end smallexample +@end deffn + +@deffn {notify-event} event @var{eid} +Trigger the notification when the event identified by @var{eid} +occurs. The identified @var{eid} is one of the following: + +@table @asis +@kwindex success +@item success + Successful upload. + +@kwindex bad-ownership +@item bad-ownership + An unauthorized user attempted to upload files for their project. + +@kwindex bad-directive-signature +@item bad-directive-signature + The directive signature does not match the public key of the +uploader. + +@kwindex bad-detached-signature +@item bad-detached-signature + The detached signature does not match the public key of the +uploader. + +@kwindex check-failure +@item check-failure + Distribution verification failed. @xref{verification}, for a +detailed description. +@end table +@end deffn + +@deffn {notify-event} module @var{modname} +Identify the module responsible for the notification. The +@var{modname} argument must have been previously initialized in a +@code{module} statement (@pxref{modules}). +@end deffn + +@deffn {notify-event} module-config @{ ... @} +This block provides module-specific configuration for @var{modname}. +Its content depends on the module used for notification. The version +@value{VERSION} of @command{wydawca} is shipped with two notification +modules: @code{mod_mailutils} for notifications via electronic mail, +and @code{mod_logstat} for logging the information via @code{syslog}. +These modules are described in detail later. +@end deffn + +@node mod_mailutils +@subsection @command{mod_mailutils}-- Mail Notification +@cindex mail notification + +Mail notification is configured using the @command{mod_mailutils} +module. To load the module, add the following statement: + +@example +@group +module mailutils mod_mailutils.so; +@end group +@end example + +The @code{module-init} section can contain the following statements: + +@deffn {mod_mailutils} from-address address Set sender address for outgoing mails. E.g.: @smallexample @@ -2279,7 +2457,7 @@ constructed from the name of the user @command{wydawca} runs as (@pxref{user privileges}) and the full domain name of the machine it runs at. -@deffn {Config} admin-address email +@deffn {mod_mailutils} admin-address email Sets the admin email address or addresses. The statistics notifications and any notifications configured to be sent to admins will be forwarded to this address. The @var{email} argument is either @@ -2306,18 +2484,19 @@ admin-address (root@@gnu.org.ua, ftp-adm@@gnu.org.ua); @menu * mailer:: * templates:: -* statreports:: -* event notification:: +* statreports:: +* mail-config:: @code{module-config} for @command{mod_mailutils} +* mod_mailutils example:: Example of @command{mod_mailutils} configuration @end menu @node mailer -@subsection Mailer +@subsubsection Mailer @cindex mailer -To send messages, @command{wydawca} uses a special logical entity -called a @dfn{mailer}. It is set in the configuration file using -@code{mailer} keyword. +To send messages, @command{mod_mailutils} uses a special logical entity +called a @dfn{mailer}. It is set in the @code{module-init} block using +the @code{mailer} keyword. -@deffn {Config} mailer url +@deffn {mod_mailutils} mailer url Set mailer @acronym{URL}. @end deffn @@ -2415,7 +2594,7 @@ mailer "|/bin/nullmail localhost -F$@{sender@} $@{rcpt@}" @end table @node templates -@subsection Message Templates +@subsubsection Message Templates @cindex templates, notification messages @cindex notification message template @cindex message template @@ -2425,7 +2604,7 @@ The message text may be specified either in place within the configuration directive it belongs to (@pxref{notification}), or defined by @code{define-message} statement. -@deffn {Config} define-message name text +@deffn {mod_mailutils} define-message name text Define message @var{name} to be @var{text}. This message can be referred to from other configuration statements by @code{@@@var{name}} notation. @@ -2457,10 +2636,11 @@ EOT; @end smallexample @node statreports -@subsection Statistic Reports +@subsubsection Statistic Reports -@deffn {Config} mail-statistics @{ @dots{} @} -The @code{mail-statistics} statement configures the statistic reports +@deffn {mod_mailutils} mail-statistics @{ @dots{} @} +The @code{mail-statistics} statement in the @code{module-init} section +for @command{mod_mailutils} configures the statistic reports sent to the system administrator. @smallexample @@ -2472,7 +2652,7 @@ mail-statistics @{ @end smallexample @end deffn -@deffn {Config: mail-statistics} message text-or-id +@deffn {mail-statistics} message text-or-id Define the message text. The argument is either the message text template, or a reference to a template previously defined by a @code{define-message} (@pxref{templates}). The reference syntax is: @@ -2485,7 +2665,7 @@ message @@@var{name}; where @var{name} is the message name as used in @code{define-message}. @end deffn -@deffn {Config: mail-statistics} statistics item-list +@deffn {mail-statistics} statistics item-list The argument is a list of statistics keywords as described in @ref{statistics}. A report will be sent only if statistics counters for at least one of the requested categories are not @@ -2498,14 +2678,14 @@ statistics (errors, access-violations, bad-signatures); @end smallexample @end deffn -@deffn {Config: mail-statistics} gpg-sign key +@deffn {mail-statistics} gpg-sign key If this statement is present, the message will be signed using the supplied @acronym{GPG} @var{key}. The key is looked up in the @acronym{GPG} home directory (@pxref{gpg-homedir}). @end deffn The statistics message is sent to addresses configured by -@code{admin-address} statement (@pxref{notification, admin-address}). +@code{admin-address} statement (@pxref{mod_mailutils, admin-address}). @cindex meta-variables in admin notifications The meta-variables available for use in statistics reports are: @@ -2612,59 +2792,34 @@ EOT; @} @end smallexample -@node event notification -@subsection Event Notification -@cindex event notification +@node mail-config +@subsubsection @code{module-config} for @command{mod_mailutils} - A number of @dfn{events} are tracked during the execution. Any of -them can be used to trigger an email notification of any party -concerned: the system administrator, project administrators, -the user that initiated the upload, or any other recipients, specified -by their email addresses. These notifications are configured -using the @code{notify-event} statement: +When @command{mod_mailutils} is used in the @command{notify-event} +block, the following statements can be used in @code{module-config} +to configure it: -@deffn {Config} notify-event @{ @dots{} @} @smallexample +@group notify-event @{ - event @var{ev-id}; - recipient @var{who}; - message @var{text-or-id}; - gpg-key @var{key}; + module mailutils; + # module configuration + module-config @{ + # Notify this recipient + recipient @var{who}; + + # Sign message with this key + gpg-sign @var{key}; + + # Text of the notification or identifier of a defined message + # template + message @var{text-or-id}; + @} @} +@end group @end smallexample -@end deffn - -@deffn {Config: notify-event} event ev-id -Send notification when the event @var{ev-id} occurs. The following -table describes the available @var{ev-id}s: -@table @asis -@kwindex success -@item success - Successful upload. - -@kwindex bad-ownership -@item bad-ownership - An unauthorized user attempted to upload files for their project. - -@kwindex bad-directive-signature -@item bad-directive-signature - The directive signature does not match the public key of the -uploader. - -@kwindex bad-detached-signature -@item bad-detached-signature - The detached signature does not match the public key of the -uploader. - -@kwindex check-failure -@item check-failure - Distribution verification failed. @xref{verification}, for a -detailed description. -@end table -@end deffn - -@deffn {Config: notify-event} recipient who +@deffn {mod_mailutils config} recipient who Determines who should receive the notification. The following values for @var{who} are allowed: @@ -2693,40 +2848,19 @@ User name of the user who uploaded files. @end table @end deffn -@deffn {Config: notify-event} message text-or-id -Define the message text. The argument is either the message text -template, or a reference to a template previously defined by a -@code{define-message} (@pxref{templates}). -@end deffn - @deffn {Config: notify-event} gpg-sign key If this statement is present, the message will be signed using the supplied @acronym{GPG} @var{key}. The key is looked up in the @acronym{GPG} home directory (@pxref{gpg-homedir}). @end deffn - For example, the following two statements instruct @command{wydawca} -to email notifications about any @code{bad-directive-signature} event to -project administrators and to the user who did the upload, using two -different templates: - -@smallexample -notify-event @{ - event bad-directive-signature; - recipient user; - message @@usermsg; -@} - -notify-event @{ - event bad-directive-signature; - recipient owner; - message @@ownermsg; -@} -@end smallexample - - The following macro-variables may be used in templates for these -notifications: +@deffn {mod_mailutils config} message text-or-id +Define the message text. The argument is either the message text +template, or a reference to a template previously defined by a +@code{define-message} (@pxref{templates}). +@end deffn + The following macro-variables are expanded in the message texts: @multitable @columnfractions 0.30 0.70 @headitem Variable @tab Replaced with @kwindex project @@ -2851,28 +2985,96 @@ newlines after the full listing: one produced by @samp{$@{triplet:ls:upload@}} and the second one taken verbatim from the message template. -@smallexample -notify-event @{ - event success; - recipient user; - message <