diff options
author | Sergey Poznyakoff <gray@gnu.org> | 2019-07-22 12:22:44 +0300 |
---|---|---|
committer | Sergey Poznyakoff <gray@gnu.org> | 2019-07-22 12:22:44 +0300 |
commit | 0917d1276103d9cc893e8ac091e9e63c5e6182f8 (patch) | |
tree | e7dc735efc3b6543922e56d24e1e3289a25e71cd /doc | |
parent | 77f10fa54db3a4f0d9c782e8e9dd39051afc19f0 (diff) | |
download | wydawca-0917d1276103d9cc893e8ac091e9e63c5e6182f8.tar.gz wydawca-0917d1276103d9cc893e8ac091e9e63c5e6182f8.tar.bz2 |
Document the module subsystem
Diffstat (limited to 'doc')
-rw-r--r-- | doc/wydawca.texi | 579 |
1 files changed, 407 insertions, 172 deletions
diff --git a/doc/wydawca.texi b/doc/wydawca.texi index ca58d1d..db06139 100644 --- a/doc/wydawca.texi +++ b/doc/wydawca.texi @@ -37,13 +37,10 @@ Copyright @copyright{} 2007, 2009, 2010, 2011, 2012, 2013, 2014, 2015, Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no -Invariant Sections, with the Front-Cover texts being ``A GNU Manual'', -and with the Back-Cover Texts as in (a) below. A copy of the license -is included in the section entitled ``GNU Free Documentation License''. +Invariant Sections, no Front-Cover, and no Back-Cover texts. -(a) The FSF's Back-Cover Text is: ``You have freedom to copy and modify -this GNU Manual, like GNU software. Copies published by the Free -Software Foundation raise funds for GNU development.'' +A copy of the license is included in the section entitled ``GNU Free +Documentation License''. @end copying @titlepage @@ -144,7 +141,7 @@ Mail Notification @chapter Introduction to Wydawca @cindex introduction Let's begin with a short synopsis. Suppose you run a developer's -site, like, e.g. @indicateurl{gnu.org}. You have two +site, such as e.g. @indicateurl{gnu.org}. You have two @dfn{distribution @acronym{URL}s}: @indicateurl{ftp.gnu.org}, which distributes stable versions of the software, and @indicateurl{alpha.gnu.org}, which distributes alpha and pre-test @@ -194,8 +191,8 @@ their distribution sites. @command{Wydawca} is such a release submission daemon. It is able to handle any number of @samp{source/destination} pairs (called -@dfn{spools}) in real time, and offers an extensible logging and mail -notification mechanism, allowing both package maintainers and site +@dfn{spools}) in real time, and offers extensible logging and +notification mechanisms, allowing both package maintainers and site administrators to be immediately notified about any occurring problems. @command{Wydawca} supports upload directive versions 1.1@footnote{ @@ -249,14 +246,14 @@ that case. upload and corresponding distribution directories. In @command{wydawca} terminology, upload directories are also called @dfn{source}, and distribution directories -- @dfn{destination} -directories. The configuration file also supplies all the information +directories. The configuration file supplies also the information necessary to access user and project databases. When started, @command{wydawca} scans each source directory and prepares a list of files found there. Then, it compacts this list by looking for @dfn{directive files} and re-arranging list members in @dfn{triplets}. A @dfn{directive file} is a special file that must be -supplied with each upload and that contains directive regarding the +supplied with each upload and that contains instructions regarding the placement of the uploaded files. A @dfn{triplet} is a standard entity, consisting of three files: a clear-signed directive file, a file to be distributed, and a detached signature of the latter. @@ -288,8 +285,8 @@ possibility to restart interrupted or otherwise broken uploads later. @cindex @acronym{PGP} After completing these preliminary stages, @command{wydawca} analyzes the directive file and extracts the project name -from it. Using this name as a key, it looks up in the @dfn{project -dictionary} a list of users authorized to make uploads for this +from it. Using this name as a key, it searches in the @dfn{project +dictionary} for a list of users authorized to make uploads for this project. This list contains user names and their corresponding public @acronym{PGP} keys. @command{Wydawca} tries to verify the directive file using each @acronym{PGP} key from this list, until a matching @@ -435,6 +432,11 @@ by using the following options: $ wydawca --spool=ftp --spool=test --source=/home/ftp/test-upload @end smallexample + Any non-optional arguments appearing in the command line, are +treated as the uploaders' UIDs. If present, only triplets uploaded by +these users will be processed. Of course, such UIDs can be combined +with the @option{--spool} and @option{--source} options. + @anchor{debug} @xopindex{debug, described} @sopindex{d, described} @@ -480,7 +482,7 @@ $ wydawca -c new.cfg --dry-run @dfn{configuration file} @file{wydawca.rc}. By default it is located in @var{$sysconfidr} (i.e., in most cases @file{/usr/local/etc}, or @file{/etc}), but an alternative location may be specified using -@option{--config} command line option (@pxref{starting, config-file}). +@option{--config-file} command line option (@pxref{starting, config-file}). If any errors are encountered in the configuration file, the program reports them on its error output and exits with a non-zero status. @@ -2252,20 +2254,196 @@ statistics (all, errors, warnings); @end deffn @node notification -@section Mail Notification -@cindex mail notification +@section Notification Mechanism +@cindex notification While running, @command{wydawca} keeps track of certain events occurring, such as, for example, broken @acronym{PGP} signatures or -file uploads attempted by unauthorized users. The utility can notify, -via email, project administrators about any of those events that -concern their projects. Additionally, the system administrator can -choose to obtain via email the execution statistics, described in the -previous section. +file uploads attempted by unauthorized users. It can issue +notifications about such events using the supplied loadable modules. + +Configuration of notifications consists of two parts. First the +required loadable module must be loaded and configured. Then, +configure the notification itself. + +@menu +* modules:: +* event notification:: +* mod_mailutils:: Mail Notification +* mod_logstat:: Notification to syslog +@end menu + +@node modules +@subsection modules + +A @dfn{loadable module} is a piece of software that provides +notification mechanism for @command{wydawca}. It is built as a UNIX +dynamically loaded library and placed in one of the preconfigured +directories which constitute a @dfn{library load path}. To load a +module, the following statement is used: + +@deffn {Config} module @var{name} @var{file} +Load the module @var{name} from @var{file}. Other places of the +configuration file can refer to the module as @var{name}. + +The @var{file} argument is a file name of the module (normally, a +@samp{file.so} or @samp{file.la} file). +@end deffn + +Unless @var{file} in the @samp{module} statement it is an absolute +file name, it will be searched in the library load path, which is +defined as: + +@enumerate 1 +@item +Optional @dfn{prefix} search directories specified by the +@samp{module-prepend-load-path} directive (see below). + +@item +@command{Wydawca} module directory: @samp{@var{$prefix}/lib/wydawca}. + +@item +Additional search directories specified by the @command{module-load-path} +directive (see below). + +@item +The value of the environment variable @env{LTDL_LIBRARY_PATH}. + +@item +The system dependent library search path (e.g. on GNU/Linux it is +defined by the file @file{/etc/ld.so.conf} and the environment variable +@env{LD_LIBRARY_PATH}). +@end enumerate + +The value of @env{LTDL_LIBRARY_PATH} and @env{LD_LIBRARY_PATH} must be a +colon-separated list of absolute directory names, for example +@samp{/usr/lib/mypkg:/lib/foo}. + +In any of these directories, @command{wydawca} first attempts to find +and load the given filename. If this fails, it tries to append the following +suffixes to it: + +@enumerate 1 +@item the libtool archive suffix: @samp{.la} + +@item the suffix used for native dynamic libraries on the host platform, +e.g., @samp{.so}, @samp{.sl}, etc. +@end enumerate + +The statements that modify the module search path are: + +@deffn {Config} module-load-path @var{list} +This directive adds the directories listed in its argument to the +module load path. Example: + +@example +module-load-path (/usr/lib/wydawca,/usr/local/wydawca/lib); +@end example +@end deffn + +@deffn {Config} module-prepend-load-path @var{list} +Same as above, but the directories from @var{list} are added to the +beginning of the module search list, rather than to its end. The +order of directories in @var{list} is preserved in both cases. +@end deffn + +Once loaded, the module can be initialized. This is done by the +following block statement: + +@deffn {Config} module-init @var{name} @{ ... @} +Initialize the module identified by @var{name}. The module must have +been previously loaded using the @samp{module} statement, as described +above. The statements between curly braces are module-specific +configuration statements. See the module descriptions below for a +detailed discussion of these. +@end deffn + +@node event notification +@subsection Event Notification + + A number of @dfn{events} are tracked during the execution. Any of +them can be used to trigger the notification mechanism. It is +configured using the following statement: + +@deffn {Config} notify-event @{ ... @} +@smallexample +@group +notify-event @{ + # Event on which to notify + event @var{eid}; - The sender email address for these notifications is set using the -@code{from-address} statement. + # Name of the module to invoke on event + module @var{modname}; -@deffn {Config} from-address address + # Module-specific configuration data + module-config @{ + ... + @} +@} +@end group +@end smallexample +@end deffn + +@deffn {notify-event} event @var{eid} +Trigger the notification when the event identified by @var{eid} +occurs. The identified @var{eid} is one of the following: + +@table @asis +@kwindex success +@item success + Successful upload. + +@kwindex bad-ownership +@item bad-ownership + An unauthorized user attempted to upload files for their project. + +@kwindex bad-directive-signature +@item bad-directive-signature + The directive signature does not match the public key of the +uploader. + +@kwindex bad-detached-signature +@item bad-detached-signature + The detached signature does not match the public key of the +uploader. + +@kwindex check-failure +@item check-failure + Distribution verification failed. @xref{verification}, for a +detailed description. +@end table +@end deffn + +@deffn {notify-event} module @var{modname} +Identify the module responsible for the notification. The +@var{modname} argument must have been previously initialized in a +@code{module} statement (@pxref{modules}). +@end deffn + +@deffn {notify-event} module-config @{ ... @} +This block provides module-specific configuration for @var{modname}. +Its content depends on the module used for notification. The version +@value{VERSION} of @command{wydawca} is shipped with two notification +modules: @code{mod_mailutils} for notifications via electronic mail, +and @code{mod_logstat} for logging the information via @code{syslog}. +These modules are described in detail later. +@end deffn + +@node mod_mailutils +@subsection @command{mod_mailutils}-- Mail Notification +@cindex mail notification + +Mail notification is configured using the @command{mod_mailutils} +module. To load the module, add the following statement: + +@example +@group +module mailutils mod_mailutils.so; +@end group +@end example + +The @code{module-init} section can contain the following statements: + +@deffn {mod_mailutils} from-address address Set sender address for outgoing mails. E.g.: @smallexample @@ -2279,7 +2457,7 @@ constructed from the name of the user @command{wydawca} runs as (@pxref{user privileges}) and the full domain name of the machine it runs at. -@deffn {Config} admin-address email +@deffn {mod_mailutils} admin-address email Sets the admin email address or addresses. The statistics notifications and any notifications configured to be sent to admins will be forwarded to this address. The @var{email} argument is either @@ -2306,18 +2484,19 @@ admin-address (root@@gnu.org.ua, ftp-adm@@gnu.org.ua); @menu * mailer:: * templates:: -* statreports:: -* event notification:: +* statreports:: +* mail-config:: @code{module-config} for @command{mod_mailutils} +* mod_mailutils example:: Example of @command{mod_mailutils} configuration @end menu @node mailer -@subsection Mailer +@subsubsection Mailer @cindex mailer -To send messages, @command{wydawca} uses a special logical entity -called a @dfn{mailer}. It is set in the configuration file using -@code{mailer} keyword. +To send messages, @command{mod_mailutils} uses a special logical entity +called a @dfn{mailer}. It is set in the @code{module-init} block using +the @code{mailer} keyword. -@deffn {Config} mailer url +@deffn {mod_mailutils} mailer url Set mailer @acronym{URL}. @end deffn @@ -2415,7 +2594,7 @@ mailer "|/bin/nullmail localhost -F$@{sender@} $@{rcpt@}" @end table @node templates -@subsection Message Templates +@subsubsection Message Templates @cindex templates, notification messages @cindex notification message template @cindex message template @@ -2425,7 +2604,7 @@ The message text may be specified either in place within the configuration directive it belongs to (@pxref{notification}), or defined by @code{define-message} statement. -@deffn {Config} define-message name text +@deffn {mod_mailutils} define-message name text Define message @var{name} to be @var{text}. This message can be referred to from other configuration statements by @code{@@@var{name}} notation. @@ -2457,10 +2636,11 @@ EOT; @end smallexample @node statreports -@subsection Statistic Reports +@subsubsection Statistic Reports -@deffn {Config} mail-statistics @{ @dots{} @} -The @code{mail-statistics} statement configures the statistic reports +@deffn {mod_mailutils} mail-statistics @{ @dots{} @} +The @code{mail-statistics} statement in the @code{module-init} section +for @command{mod_mailutils} configures the statistic reports sent to the system administrator. @smallexample @@ -2472,7 +2652,7 @@ mail-statistics @{ @end smallexample @end deffn -@deffn {Config: mail-statistics} message text-or-id +@deffn {mail-statistics} message text-or-id Define the message text. The argument is either the message text template, or a reference to a template previously defined by a @code{define-message} (@pxref{templates}). The reference syntax is: @@ -2485,7 +2665,7 @@ message @@@var{name}; where @var{name} is the message name as used in @code{define-message}. @end deffn -@deffn {Config: mail-statistics} statistics item-list +@deffn {mail-statistics} statistics item-list The argument is a list of statistics keywords as described in @ref{statistics}. A report will be sent only if statistics counters for at least one of the requested categories are not @@ -2498,14 +2678,14 @@ statistics (errors, access-violations, bad-signatures); @end smallexample @end deffn -@deffn {Config: mail-statistics} gpg-sign key +@deffn {mail-statistics} gpg-sign key If this statement is present, the message will be signed using the supplied @acronym{GPG} @var{key}. The key is looked up in the @acronym{GPG} home directory (@pxref{gpg-homedir}). @end deffn The statistics message is sent to addresses configured by -@code{admin-address} statement (@pxref{notification, admin-address}). +@code{admin-address} statement (@pxref{mod_mailutils, admin-address}). @cindex meta-variables in admin notifications The meta-variables available for use in statistics reports are: @@ -2612,59 +2792,34 @@ EOT; @} @end smallexample -@node event notification -@subsection Event Notification -@cindex event notification +@node mail-config +@subsubsection @code{module-config} for @command{mod_mailutils} - A number of @dfn{events} are tracked during the execution. Any of -them can be used to trigger an email notification of any party -concerned: the system administrator, project administrators, -the user that initiated the upload, or any other recipients, specified -by their email addresses. These notifications are configured -using the @code{notify-event} statement: +When @command{mod_mailutils} is used in the @command{notify-event} +block, the following statements can be used in @code{module-config} +to configure it: -@deffn {Config} notify-event @{ @dots{} @} @smallexample +@group notify-event @{ - event @var{ev-id}; - recipient @var{who}; - message @var{text-or-id}; - gpg-key @var{key}; + module mailutils; + # module configuration + module-config @{ + # Notify this recipient + recipient @var{who}; + + # Sign message with this key + gpg-sign @var{key}; + + # Text of the notification or identifier of a defined message + # template + message @var{text-or-id}; + @} @} +@end group @end smallexample -@end deffn - -@deffn {Config: notify-event} event ev-id -Send notification when the event @var{ev-id} occurs. The following -table describes the available @var{ev-id}s: -@table @asis -@kwindex success -@item success - Successful upload. - -@kwindex bad-ownership -@item bad-ownership - An unauthorized user attempted to upload files for their project. - -@kwindex bad-directive-signature -@item bad-directive-signature - The directive signature does not match the public key of the -uploader. - -@kwindex bad-detached-signature -@item bad-detached-signature - The detached signature does not match the public key of the -uploader. - -@kwindex check-failure -@item check-failure - Distribution verification failed. @xref{verification}, for a -detailed description. -@end table -@end deffn - -@deffn {Config: notify-event} recipient who +@deffn {mod_mailutils config} recipient who Determines who should receive the notification. The following values for @var{who} are allowed: @@ -2693,40 +2848,19 @@ User name of the user who uploaded files. @end table @end deffn -@deffn {Config: notify-event} message text-or-id -Define the message text. The argument is either the message text -template, or a reference to a template previously defined by a -@code{define-message} (@pxref{templates}). -@end deffn - @deffn {Config: notify-event} gpg-sign key If this statement is present, the message will be signed using the supplied @acronym{GPG} @var{key}. The key is looked up in the @acronym{GPG} home directory (@pxref{gpg-homedir}). @end deffn - For example, the following two statements instruct @command{wydawca} -to email notifications about any @code{bad-directive-signature} event to -project administrators and to the user who did the upload, using two -different templates: - -@smallexample -notify-event @{ - event bad-directive-signature; - recipient user; - message @@usermsg; -@} - -notify-event @{ - event bad-directive-signature; - recipient owner; - message @@ownermsg; -@} -@end smallexample - - The following macro-variables may be used in templates for these -notifications: +@deffn {mod_mailutils config} message text-or-id +Define the message text. The argument is either the message text +template, or a reference to a template previously defined by a +@code{define-message} (@pxref{templates}). +@end deffn + The following macro-variables are expanded in the message texts: @multitable @columnfractions 0.30 0.70 @headitem Variable @tab Replaced with @kwindex project @@ -2851,28 +2985,96 @@ newlines after the full listing: one produced by @samp{$@{triplet:ls:upload@}} and the second one taken verbatim from the message template. -@smallexample -notify-event @{ - event success; - recipient user; - message <<EOT -Subject: Upload of $@{project@} successful - -Upload of $@{project@} to $@{url@}/$@{dir@} finished successfully. -Files uploaded: +@node mod_mailutils example +@subsubsection Example of mod_mailutils configuration -$@{triplet:ls:upload@}$- +This subsection provides a complete example for +@command{mod_mailutils} configuration. -Resource usage: $@{timer:triplet:real@}/$@{timer:wydawca:real@}r \ -$@{timer:triplet:user@}/$@{timer:wydawca:user@}u \ -$@{timer:triplet:system@}/$@{timer:wydawca:system@}s +@example +module mailutils mod_mailutils.la; + +module-init mailutils @{ + admin-address "root@@example.net"; + from-address "wydawca@@example.net"; + mailer "sendmail:"; + + mail-statistics @{ + statistics all; + message <<- EOT + Subject: upload statistics + + This is to notify you that the run of wydawca on $@{date@} + caused the following results: + + errors ............................. $@{stat:errors@} + warning ............................ $@{stat:warnings@} + bad signatures ..................... $@{stat:bad_signatures@} + access violation attempts .......... $@{stat:access_violations@} + complete triplets .................. $@{stat:complete_triplets@} + incomplete triplets ................ $@{stat:incomplete_triplets@} + bad triplets ....................... $@{stat:bad_triplets@} + expired triplets ................... $@{stat:expired_triplets@} + triplet successes .................. $@{stat:triplet_success@} + files uploaded ..................... $@{stat:uploads@} + files archived ..................... $@{stat:archives@} + symlinks created ................... $@{stat:symlinks@} + symlinks removed ................... $@{stat:rmsymlinks@} + verification failures .............. $@{stat:check-failures@} + + Timings: + + Real ............................... $@{timer:wydawca:real@} \ + ($@{timer:releases:real@} + \ + $@{timer:alpha:real@} + \ + $@{timer:test:real@}) + System ............................. $@{timer:wydawca:system@} \ + ($@{timer:releases:system@} + \ + $@{timer:alpha:system@} + \ + $@{timer:test:system@}) + User ............................... $@{timer:wydawca:user@} \ + ($@{timer:releases:user@} + \ + $@{timer:alpha:user@} + \ + $@{timer:test:user@}) + + Regards, + Wydawca + EOT; + @} +@} -Regards, -Wydawca -The Project Submission Robot -EOT; +notify-event @{ + event success; + module mailutils; + module-config @{ + recipient user; + message <<- EOT + Subject: Upload of $@{project@} successful + + Upload of $@{project@} to $@{url@}/$@{dir@} finished successfully. + Files uploaded: + + $@{triplet:ls:upload@}$- + + Resource usage: $@{timer:triplet:real@}/$@{timer:wydawca:real@}r \ + $@{timer:triplet:user@}/$@{timer:wydawca:user@}u \ + $@{timer:triplet:system@}/$@{timer:wydawca:system@}s + + Regards, + Wydawca + The Project Submission Robot + EOT; + @} @} -@end smallexample +@end example + +For the sake of brevity, this example defines only one +@code{notify-event} statement. More statements for others events can +be added as needed. + +@node mod_logstat +@subsection Notification to syslog +@UNREVISED @node wydawca.rc @chapter @command{Wydawca} configuration file. @@ -2953,21 +3155,84 @@ tcp-wrapper @{ deny-syslog-priority @var{prio:@i{string}}; @} -# @r{Set mailer URL.} -# @xref{mailer}. -mailer @var{url:@i{string}}; +# @r{Load module} +# @xref{modules}. +module @var{name:@i{string}} @var{file:@i{string}}; + +# @r{Initialize the loaded module} +# @xref{modules,module-init}. +module-init @var{name} @{ + # @r{One or more module-specific statements} + ... +@} + +# @r{Load mailutils module} +# @xref{mod_mailutils}. +module mailutils mod_mailutils.so; + +# @r{mod_mailutils initialization} +module-init mailutils @{ + # @r{Set mailer URL.} + # @xref{mailer}. + mailer @var{url:@i{string}}; + + # @r{Set admin email address.} + # @xref{mod_mailutils, admin-address}. + admin-address @var{email:@i{string}}; + + # @r{Set sender email address.} + # @xref{mod_mailutils, from-address}. + from-address @var{email:@i{string}}; + + # @r{Send statistics report.} + # @xref{statreports}. + mail-statistics @{ + # @r{Message text.} + message @var{text:@i{string}}; + + # @r{Send mail if one or more of these items are set.} + statistics @var{items:@i{string}}; -# @r{Set admin email address.} -# @xref{notification, admin-address}. -admin-address @var{email:@i{string}}; + # @r{Sign message with this key.} + gpg-sign @var{key:@i{string}}; + @} +@} + +# @r{Configure notification.} +# @xref{notification}. +notify-event @{ + # @r{Event on which to notify.} + event @var{ev-id:@i{string}}; -# @r{Set sender email address.} -# @xref{notification, from-address}. -from-address @var{email:@i{string}}; + # Name of the module to invoke on event + module @var{modname:@i{string}}; + + # Module-specific configuration data + module-config @{ + ... + @} + + # Configuration for mod_mailutils + # @xref{mail-config}. + module-config @{ + # Notify this recipient + # @xref{mail-config, recipient}. + recipient @var{who:@i{string}}; + + # Sign message with this key + # @xref{mail-config, gpg-sign}. + gpg-sign @var{key:@i{string}}; + + # Text of the notification or identifier of a defined message + # template + # @xref{mail-config, message}. + message @var{text-or-id:@i{string}}; + @} +@} # @r{Define file sweep time.} # @xref{general, file-sweep-time}. -file-sweep-time @var{time:@i{interval}}; +file-sweep-time @var{time:@i{int9erval}}; # @r{Set tar invocation command line.} # @xref{archivation, tar-program}. @@ -3045,36 +3310,6 @@ archive @var{type:@i{string}} @{ backup @var{type:@i{string}}; @} -# @r{Send statistics.} -# @xref{statreports}. -mail-statistics @{ - # @r{Message text.} - message @var{text:@i{string}}; - - # @r{Send mail if one or more of these items are set.} - statistics @var{items:@i{string}}; - - # @r{Sign message with this key.} - gpg-sign @var{key:@i{string}}; -@} - -# @r{Configure notification.} -# @xref{notification}. -notify-event @{ - # @r{Event on which to notify.} - event @var{ev-id:@i{string}}; - - # @r{Notify this recipient.} - recipient @var{who:@i{string}}; - - # @r{Text of the notification or identifier of} - # @r{a defined message template.} - message @var{text-or-id:@i{string}}; - - # @r{Sign message with this key.} - gpg-sign @var{key:@i{string}}; -@} - # @r{Define data dictionary.} # @xref{dictionaries}. dictionary @var{ident:@i{string}} @{ |