     [file version.texi not included]

Copyright © 2013-2014 Sergey Poznyakoff

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, and with Front-Cover and Back-Cover at your option. A copy of the license is included in the section entitled ``GNU Free Documentation License''.

Direvent
********

This edition of the GNU Direvent manual, last updated UPDATED, documents GNU Direvent Version VERSION.

Introduction
************

GNU direvent monitors events in file system directories.  For each event that occurs in a set of pre-configured directories, the program calls an external program associated with it, supplying it the information about the event and the location within the file system where it took place.

GNU direvent provides an easy way to configure your system to react immediately if certain files undergo changes. This may be helpful, for example, to track changes in important configuration files.

Interfaces for tracking changes to file systems are highly system-specific.  GNU direvent aims to provide a uniform and system-independent command-level interface.  As of version VERSION direvent works with modern Linux kernels (since v. 2.6.13) and BSD systems (FreeBSD, NetBSD, OpenBSD, Darwin).

Overview
********

GNU direvent monitors a set of directories on the file system and reacts when a file system event occurs in any of them.  Directories and events to monitor are specified in the configuration file.  When an event occurs, the program reacts by invoking an external command configured for that event.

File system events can be divided into two major groups.  The system-dependent events are specific for each particular kernel interface.  In the contrast, generic events don't depend on the underlying system.  They provide a higher level of abstraction and make it possible to port GNU direvent configurations between various systems and architectures.

The generic events are:

create
     A file was created.

delete
     A file was deleted;

write
     A file was written to;

attrib
     File attributes have changed.  This includes changes in the file ownership, mode, link count, etc.

A watcher is a configuration entity that associates a set of directories with a set of events and instructs direvent to run a specified external command when any of these events occur in any of these directories.  This external command (called a handler) can obtain information about the event that triggered it from the environment variables, or from its command line.

Watchers are defined in the configuration file, which direvent reads at startup.  This file has a simple and easy to use syntax.

Three types of comments are allowed: inline comments, that begin with a # or // and extend to the end of line, and multi-line comments, which comprise everything enclosed between /* and */.  Comments and empty lines are ignored.  Whitespace characters are ignored as well, except as they serve to separate tokens.

A token is a string of consecutive characters from the following classes: alphanumeric characters, underscores, dots, asteriscs, slashes, semicolons, commercial at's, and dashes.

Any other sequence of characters must be enclosed in double quotation marks in order to represent a single token.

Adjacent quoted strings are concatenated.

A configuration statement consists of a keyword and value separated by any amount of whitespace and is terminated with a semicolon.  A block statement is a collection of statements enclosed in curly braces.

The following block statement declares a watcher:

     watcher {
         path PATHNAME [recursive [LEVEL]];
         file PATTERN-LIST;
         event  EVENT-LIST;
         command COMMAND-LINE;
         user NAME;
         timeout NUMBER;
         environ ENV-SPEC;
         option STRING-LIST;
     }

Each watcher statement instructs direvent to monitor events listed in EVENT-LIST occurring in the directories specified by PATHNAMEs in path statements (any number of path statements can be given).  When any such event is detected, the supplied COMMAND-LINE will be executed.

Each directory defined with the recursive keyword will be watched recursively.  This means that for each subdirectory created in it, direvent will install a watcher similar to that of its parent directory.  Optional LEVEL statement can be used to set up a cut-off nesting level, beyond which the recursive operation is disabled.

The rest of statements are optional.  The file statement instructs GNU direvent to react only if the event concerned the file whose name matches one of the patterns given in its argument. The user statement can be used to execute the COMMAND-LINE as the user NAME (provided, of course, that direvent is started with root privileges).  The timeout specifies the maximum amount of time (in seconds) the command is allowed to run.  It defaults to 5.  The environ statement modifies the command environment.  Finally, the option statement supplies additional options.  It can be used, for example, to divert the command's output to syslog.

Quick Start
***********

Let's suppose you have a directory where users can upload their files and you want these files to be processed right after upload, in real time.  Let this directory be /home/ftp/incoming and the program to process the upload be /usr/bin/upload.  Let's also suppose that this program expects name of the uploaded file as its argument.

To make direvent handle this task, you would need to create a watcher for the upload directory which would handle the create event:

     watcher {
         path /home/ftp/incoming;
         event create;
         # more statements follow...

On this event, the watcher is to invoke /usr/bin/upload with the name of the created file as an argument.  To make it possible, the direvent configuration file provides macro variables, which can be used in the command argument at configuration time and which are expanded to the actual values before the command is executed.  Macro variables are referred to using the same syntax as shell variables: a dollar sign followed by the variable name, optionally enclosed in curly braces.  The file variable is expanded to the name of the file for which the event is reported. This name is relative to the current working directory which, by the time the handler is executed, is set to the directory where the event occurred.  Thus, the handler can be configured as:

         command "/usr/bin/upload $file";

To summarize, the watcher declaration is:

     watcher {
         path /home/ftp/incoming;
         event create;
         command "/usr/bin/upload $file";
     }

Before invoking the handler, the following operations are performed:

 1. The current working directory is set to the directory where the event occurred.

 2. If the environ statement is present in the watcher, the environment is modified according to its rules.  (see environment modification)

 3. The standard input is closed.

 4. If the stdout option is supplied, the standard output is captured and redirected to the syslog. Otherwise it is closed.

 5. If the stderr option is supplied, the standard error is captured and redirected to the syslog. Otherwise it is closed.

 6. File descriptors above 2 are closed.

 7. Macro variables are expanded.  See macro expansion.

 8. Word splitting is performed on the resulting command line.  The first word is treated as the pathname of the program to be executed.

 9. The program is invoked.

Invocation
**********

The invocation syntax is:

     direvent  [OPTIONS] [CONFIG]

where OPTIONS are command line options discussed below and optional CONFIG supplies the configuration file to use instead of the default /etc/direvent.conf.

The options are:

-d
--debug
     Increase debug level.

-F NAME
--facility=NAME
     Set syslog facility.

-f
--foreground
     Remain in foreground.

-l PRIO
     While connected to a terminal, direvent outputs its diagnostics messages to stderr and, if configured, to syslog.  This option limits the amount of information output to the standard error. The PRIO argument is one of the following priorities (in order of increasing severity): debug, info, notice, warning, err, crit, alert, emerg. When this option is given, only messages with the priority level equal to or greater than PRIO will be duplicated on the standard error.

-P FILE
--pidfile=FILE
     Upon successful startup store the PID of the daemon process in FILE.

-T COMMAND
--self-test=COMMAND
     Run in self-test mode.  In this mode, direvent starts external command supplied as the argument to this option and continues running until the command exits.  If COMMAND terminates normally, direvent exits with the code returned by it.  If COMMAND terminates on signal, direvent exits with code 0 if this signal was SIGHUP, and with code 2 otherwise.

     The COMMAND can include any command line options or arguments, provided that it is properly quoted.  It is invoked as /bin/sh -c COMMAND in the environment of the parent direvent process.

     This mode is used in direvent test suite.  The idea is to configure the handler (see handler) so that it sends SIGHUP to COMMAND before exiting.  To this effect, the special macro variable $self_test_pid is defined (see macro expansion) to the PID of the running COMMAND process.  For example, consider configuration file test.conf, which contains the following:

          watcher {
              path /tmp;
              command "/bin/kill -HUP $self_test_pid";
          }

     Then, the following command can be used to check whether direvent correctly reacts on file creation in the watched directory:

          $ direvent --foreground \
              --self-test 'touch /tmp/file && /usr/bin/sleep 20 && exit 1' \
              test.conf

     The command will return 0 if the handler was invoked, and 1 if it was not.

-t
--lint
     Check configuration file for errors and exit.

-u NAME
--user=NAME
     Run as this user.  This option overrides the user configuration statement (see user).

The following options are informative.  They cause the program to display the requested piece of information and terminate:

-H
--config-help
     Show configuration file summary.

-h
--help
     Give a short usage summary.

--usage
     Display available command line options.

-V
--version
     Print program version.

Configuration
*************

Configuration Syntax
====================

The configuration file consists of statements and comments.

There are three classes of lexical tokens: keywords, values, and separators.  Blanks, tabs, newlines and comments, collectively called white space are ignored except as they serve to separate tokens.  Some white space is required to separate otherwise adjacent keywords and values.

Comments
--------

Comments may appear anywhere where white space may appear in the configuration file.  There are two kinds of comments: single-line and multi-line comments.  Single-line comments start with # or // and continue to the end of the line:

     # This is a comment
     // This too is a comment

Multi-line or C-style comments start with the two characters /* (slash, star) and continue until the first occurrence of */ (star, slash).

Multi-line comments cannot be nested.  However, single-line comments may well appear within multi-line ones.

Pragmatic Comments
------------------

Pragmatic comments are similar to usual single-line comments, except that they cause some changes in the way the configuration is parsed.  Pragmatic comments begin with a # sign and end with the next physical newline character.

#include <FILE>
#include FILE
     Include the contents of the file FILE.

#include_once <FILE>
#include_once FILE
     Same as #include, except that, if the FILE has already been included, it will not be included again.

#line NUM
#line NUM "FILE"
     This line causes the parser to believe, for purposes of error diagnostics, that the line number of the next source line is given by NUM and the current input file is named by FILE. If the latter is absent, the remembered file name does not change.

# NUM "FILE"
     This is a special form of #line statement, understood for compatibility with the C preprocessor.

Statements
----------

A simple statement consists of a keyword and value separated by any amount of whitespace.  Simple statement is terminated with a semicolon (;).

The following is a simple statement:

     standalone yes;
     pidfile /var/run/direvent.pid;

A keyword begins with a letter and may contain letters, decimal digits, underscores (_) and dashes (-). Examples of keywords are: expression, output-file.

A value can be one of the following:

number
     A number is a sequence of decimal digits.

boolean
     A boolean value is one of the following: yes, true, t or 1, meaning true, and no, false, nil, 0 meaning false.

unquoted string
     An unquoted string may contain letters, digits, and any of the following characters: _, -, ., /, @, *, :.

quoted string
     A quoted string is any sequence of characters enclosed in double-quotes (").  A backslash appearing within a quoted string introduces an escape sequence, which is replaced with a single character according to the following rules:

     Table: Backslash escapes

     Sequence   Replaced with
     \a   Audible bell character (ASCII 7)
     \b   Backspace character (ASCII 8)
     \f   Form-feed character (ASCII 12)
     \n   Newline character (ASCII 10)
     \r   Carriage return character (ASCII 13)
     \t   Horizontal tabulation character (ASCII 9)
     \v   Vertical tabulation character (ASCII 11)
     \\   A single backslash (\)
     \"   A double-quote.

     In addition, the sequence \NEWLINE is removed from the string.  This allows to split long strings over several physical lines, e.g.:

          "a long string may be\
           split over several lines"

     If the character following a backslash is not one of those specified above, the backslash is ignored and a warning is issued.

     Two or more adjacent quoted strings are concatenated, which gives another way to split long strings over several lines to improve readability.  The following fragment produces the same result as the example above:

          "a long string may be"
          " split over several lines"

Here-document
     A here-document is a special construct that allows to introduce strings of text containing embedded newlines.

     The <<WORD construct instructs the parser to read all the following lines up to the line containing only WORD, with possible trailing blanks.  Any lines thus read are concatenated together into a single string.  For example:

          <<EOT
          A multiline
          string
          EOT

     The body of a here-document is interpreted the same way as a double-quoted string, unless WORD is preceded by a backslash (e.g.  <<\EOT) or enclosed in double-quotes, in which case the text is read as is, without interpretation of escape sequences.

     If WORD is prefixed with - (a dash), then all leading tab characters are stripped from input lines and the line containing WORD.  Furthermore, if - is followed by a single space, all leading whitespace is stripped from them.  This allows to indent here-documents in a natural fashion.  For example:

          <<- TEXT
              The leading whitespace will be
              ignored when reading these lines.
          TEXT

     It is important that the terminating delimiter be the only token on its line.  The only exception to this rule is allowed if a here-document appears as the last element of a statement.  In this case a semicolon can be placed on the same line with its terminating delimiter, as in:

          help-text <<-EOT
                  A sample help text.
          EOT;

list
     A list is a comma-separated list of values.  Lists are enclosed in parentheses.  The following example shows a statement whose value is a list of strings:

          option (stdout,stderr);

     In any case where a list is appropriate, a single value is allowed without being a member of a list: it is equivalent to a list with a single member.  This means that, e.g.

          option wait;

     is equivalent to

          option (wait);

A block statement introduces a logical group of statements.  It consists of a keyword, followed by an optional value, and a sequence of statements enclosed in curly braces, as shown in the example below:

     syslog {
       facility local0;
       tag "direvent";
     }

The closing curly brace may be followed by a semicolon, although this is not required.

Macro Expansion
===============

Arguments of some statements undergo macro expansion before use. During the macro expansion any occurrence of $NAME is replaced by the value of the macro variable NAME.  Variable names follow the usual convention: they begin with a letter and contain letters, digits and underscores.  Curly braces around the NAME are optional. They are required only if the macro reference is followed by a character that is not to be interpreted as part of its name, as in ${command}string.

The following macros are defined:

file
     Name of the file that triggered the event.

genev_code
     Generic (system-independent) event code.  It is a bitwise OR of the event codes represented as a decimal number.

genev_name
     Generic event name.  If several generic events are reported simultaneously, the value of this variable is a list of event names separated by space characters.  Each name corresponds to a bit in $genev_code.

self_test_pid
     The PID of the external command started with the --self-test option (see self-test mode).  If direvent is started without this option, this variable is not defined.

sysev_code
     A system-dependent event code.  It is a bitwise OR of the event codes represented as a decimal number.

sysev_name
     A system-dependent event name.  If several events are reported, the value of this variable is a list of event names separated by space characters.  Each name corresponds to a bit in sysev_code. See System dependencies, for a list of system-dependent event names.

General Settings
================

 -- Config: user NAME
     Sets the user to run as.  The NAME argument must be a name of an existing user.

 -- Config: foreground BOOL
     Run in foreground.

 -- Config: pidfile FILE
     Upon successful startup store the PID of the daemon process in FILE.

 -- Config: debug NUMBER
     Set debug level.  Valid NUMBER values are 0 (no debug) to 3 (maximum verbosity).

Syslog
======

While connected to the terminal, direvent outputs its diagnostics and debugging messages to the standard error.  After disconnecting from the controlling terminal it closes the first three file descriptors and directs all its output to the syslog.  When running in foreground mode, its messages are sent both to the standard error and to the syslog.

The following configuration statement controls the syslog output:

     syslog {
         facility STRING;
         tag STRING;
         print-priority BOOL;
     }

The statements are:

 -- Config: facility STRING
     Set syslog facility.  The argument is one of the following: user, daemon, auth or authpriv, mail, cron, local0 through local7 (case-insensitive), or a facility number.

 -- Config: tag STRING
     Tag syslog messages with string.  Normally the messages are tagged with the program name.

 -- Config: print-priority bool
     Prefix each message with its priority.

An example syslog statement:

     syslog {
         facility local0;
         print-priority yes;
     }

Watcher
=======

The watcher statement configures a single event watcher.  A watcher can control several events in multiple pathnames.  Any number of watcher statements is allowed in the configuration file, each one of them declaring a separate watcher.

     watcher {
         path PATHNAME [recursive [LEVEL]];
         file REGEXP-LIST;
         event EVENT-LIST;
         command COMMAND-LINE;
         user NAME;
         timeout NUMBER;
         environ ENV-SPEC;
         option STRING-LIST;
     }

The statements within a watcher block are:

 -- Config: path PATHNAME [recursive [NUMBER]]
     Defines a pathname to watch.  The PATHNAME argument must be the name of an existing directory in the file system.  The watcher will watch events occurring for all files within that directory.  If the optional recursive clause is specified, this directory will be watched recursively, i.e. when any subdirectory is created in it, direvent will set up a watcher for files in this subdirectory.  This new watcher will be an exact copy of the parent watcher, excepting for the pathnames.  The optional NUMBER parameter defines a cut-off nesting level for recursive watching.  If supplied, the recursive behaviour will apply only to the directories that are nested below that level.

     Any number of path statements can appear in a watcher block. At least one path must be defined.

 -- Config: file REGEXP-LIST
     Selects which files are eligible for monitoring.  The argument is a list of globbing patterns (in the sense of see fnmatch (fnmatch(3))) or extended regular expressions (see Extended regular expressions (GNU sed)) one of which must match the file name in order for the watcher to act on it.  A ! in front of a pattern or regular expression indicates negation.  Such construct matches if the file name doesn't match the pattern.  Regular expressions must be surrounded by a pair of slashes, optionally followed by the following flags:

     b
          Use basic regular expressions.

     i
          Enable case-insensitive matching.

     For example:

          file ("*.cfg", "/.*\\.jpg/i");

     In this statement, the first string (*.cfg) is treated as a shell globbing pattern.  The second one is a case-sensitive extended regular expression.

 -- Config: event STRING-LIST
     Configures the filesystem events to watch for in the directories declared by the path statements.  The argument is a list of event names.  Both generic and system-dependent event namess are allowed.  Multiple event statements accumulate.

     A missing event statement means ``watch all events''.

     For example:

          event (open,delete);

 -- Config: command STRING
     Defines a command to execute on event.  The STRING is a command line just as you would type it in sh.  It may contain macro variables (see macro expansion), which will be expanded prior to execution.

     For example:

          command "/bin/prog -event $genev_name -file $file";

     See handler environment, for a detailed discussion of how the command is executed.

 -- Config: user STRING
     Run command as this user.

 -- Config: timeout NUMBER
     Terminate the command if it runs longer than NUMBER seconds.  The default is 5 seconds.

 -- Config: option STRING-LIST
     A list of additional options.  The following options are defined:

     wait
          Wait for the program to terminate before handling next event from the event queue.  Normally the program runs asynchronously.

     stdout
          Capture the standard output of the command and redirect it to the syslog with the LOG_INFO priority.

     stderr
          Capture the standard error of the command and redirect it to the syslog with the LOG_ERR priority.

 -- Config: environ ENV-SPEC
     Modify command environment.  By default the command inherits the environment of direvent augmented with the following variables:

     DIREVENT_SYSEV_CODE
          The system-dependent event code (see the $sysev_code variable).

     DIREVENT_SYSEV_NAME
          The system-dependent event name or names (see the $sysev_name variable).

     DIREVENT_GENEV_CODE
          The generic event code (see the $genev_code variable).

     DIREVENT_GENEV_NAME
          The generic event name or names (see the $genev_name variable).

     DIREVENT_FILE
          The name of the affected file relative to the current working directory (see the $file variable).

     The environ statement allows for trimming the environment.  Its argument is a list of environment modification directives.  Before applying, each directive undergoes macro expansion (see macro expansion).  The following directives are available:

     - (a single dash)
          Clear the inherited environment, but retain the variables added by direvent itself.  The removed environment variables can be selectively restored using the directives discussed below.

          If used, this must be the first directive in the list.

     -- (double-dash)
          Clear the entire environment, including the variables added by direvent.

          If used, this must be the first directive in the list.

     -NAME
          Unset the variable NAME.

     -NAME=VAL
          Unset the environment variable NAME only if its value is VAL.

     NAME
          Restore the environment variable NAME.  This directive is useful after - or -- to retain some variables from the environment.

     NAME=VALUE
          Define environment variable NAME to have given VALUE.

     NAME+=VALUE
          Retain variable NAME and append VALUE to its existing value.  If no such variable is present in the environment, it is created and VALUE is assigned to it.  However, if VALUE begins with a punctuation character, this character is removed from it before the assignment.  This is convenient for using this construct with environment variables like PATH, e.g.:

               PATH+=:/sbin

          In this example, if PATH exists, :/sbin will be appended to it.  Otherwise, it will be created and /sbin will be assigned to it.

     NAME=+VALUE
          Retain variable NAME and prepend VALUE to its existing value.  If no such variable is present in the environment, it is created and VALUE is assigned to it.  However, if VALUE ends with a punctuation character, this character is removed from it before assignment.

System Dependencies
*******************

Direvent relies on the event monitoring API provided by the kernel.

GNU/Linux systems.
==================

On GNU/Linux the program uses inotify. See monitoring file system events (inotify(7) man page).

The maximum number of watches a user process can have is controlled by the fs.inotify.max_user_watches system variable.  Normally it is set to 8192, which is quite enough for most purposes.  However, if you monitor a big number or directories and/or are using recursive watchers, you may need to increase this number.  In that case, use sysctl (see configure kernel parameters at runtime (sysctl(8) man page)) to raise the limit, e.g.:

     sysctl -w fs.inotify.max_user_watches=16384

Most GNU/Linux distributions provide the file /etc/sysctl.conf which can be used to set this variable on startup.

The following system-dependent events are defined on systems that use inotify:

ACCESS
     A file was accessed.

ATTRIB
     A file's metadata changed.

CLOSE_WRITE
     A writable file was closed.

CLOSE_NOWRITE
     An unwritable file closed.

CREATE
     A file was created.

DELETE
     A file was deleted.

MODIFY
     A file was modified.

MOVED_FROM
     A file was moved into a monitored directory.

MOVED_TO
     A file was moved out from a monitored directory.

OPEN
     A file was opened.

BSD systems
===========

When compiled on BSD systems (including Darwin), direvent uses kqueue (see kernel event notification mechanism (kqueue(2) man page)).

This interface needs an open file handle for each file in a monitored directory, which means that the number of watchers is limited by the maximum number of open files.  Use ulimit -n NUM in order to raise it to a higher number.

Since it operates on files, kqueue does not provide direct support for the create generic event.  Direvent works over this disadvantage by keeping track of the contents of each monitored directory and rescanning it each time a WRITE system event is reported for it.  It then generates the open event for each file that appeared after the last scan.  Such a rescan can consume considerable time if a directory has a very large number of files in it.

The following system-dependent events are available:

DELETE
     The unlink() system call was called on the monitored file.

WRITE
     A write occurred on the file.

EXTEND
     The file was extended.

ATTRIB
     The file attributes have changed.

LINK
     The link count on the file changed.

RENAME
     The file was renamed.

REVOKE
     Access to the file was revoked via revoke() (see revoke file access (revoke(2) man page)) or the underlying file system was unmounted.

Darwin (Mac OS X)
=================

Essentially the same as BSD.  The main difference compared to Linux and BSD is that on Darwin the watchers are set after disconnecting from the controlling terminal, because Darwin lacks the rfork call and the event queue cannot be inherited by the child process.

How to Report a Bug
*******************

Please, report bugs and suggestions to bug-direvent@gnu.org.ua.

You hit a bug if at least one of the conditions below is met:

  * direvent terminates on signal 11 (SIGSEGV) or 6 (SIGABRT).

  * The program fails to do its job as described in this manual.

If you think you've found a bug, please be sure to include maximum information available to reliably reproduce it, or at least to analyze it.  The information needed is:

  * Version of the package you are using.

  * Command line options and configuration file.

  * Conditions under which the bug appears.

Any errors, typos or omissions found in this manual also qualify as bugs.  Please report them, if you happen to find any.

GNU Free Documentation License
******************************

     [file fdl.texi not included]

Concept Index
*************

This is a general index of all issues discussed in this manual