aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorFiles
2020-04-25Keep statistic items in global array. Use special thread for periodic reportingSergey Poznyakoff15
* src/config.c: New configuration statement stat-report-interval * src/directive.c * src/timer.c: Rewrite statistic counters and functions (wydawca_stat_log,wydawca_stat_init,wydawca_stat_update) (wydawca_stat_notify,wydawca_stat_add): New functions. (wy_thr_stat): New thread (statistics reporter. * src/triplet.c: Remove per-thread statistic counters. * src/wydawca.c (wy_main): Use new statistic calls. Force statistic logging at the end of the run. * src/wydawca.h (DEFAULT_STAT_REPORT_INTERVAL): New constant. (stat_report_interval): New extern. (wy_get_stat_array, wy_get_stat_slot) (wy_get_stat_counter): Remove. (wydawca_stat_log,wydawca_stat_init,wydawca_stat_update) (wydawca_stat_notify,wydawca_stat_add) (wydawca_stat_incr): New protos. (stat_mask_p,logstats,wydawca_stats_export) (wydawca_stats_update): Remove protos.
2020-04-25Fix string expansion.Sergey Poznyakoff3
* modules/mailutils/mod_mailutils.c (mail_stats): Handle NULL value. * src/triplet.c (try_timer_var): Don't exit on errors. Handle timer:spool:X:Y properly. (wy_expand_string): Don't exit on errors. Improve error reporting. * src/wydawca.c (wy_stat_expansion): Return after successful expansion.
2020-04-25Fix eventual simultaneous access to the triplet.Sergey Poznyakoff5
The triplet could have been modified by the triplet_remove_file function invoked from the watcher on file delete or moved_from event, triggered by the normal install procedure. * src/wydawca.h (struct wy_triplet): Replace the destroying member with in_processing. * src/triplet.c (triplet_enqueue): Set the "in_processing" member. (remove_triplet_unlocked): Remove the "destroying" flag. (triplet_remove_file): If in_processing is set, don't touch the triplet. * src/directive.c (process_directives): Remove the "destroying" flag. * src/timer.c (timer_stop): Use timer_get to ensure timing is computed. * src/watcher.c: Remove unused structure member.
2020-04-24Adjust thread namesSergey Poznyakoff2
2020-04-24Set thread names (and fix previos incomplete commit.Sergey Poznyakoff9
* Makefile.am: Restore ACLOCAL_AMFLAGS * configure.ac: Restore AC_CONFIG_MACRO_DIR. Call AC_USE_SYSTEM_EXTENSIONS and AX_THREAD_NAMES * am/ax_thread_names.m4: New file. * src/thread_name.c: New file. * src/Makefile.am: Add thread_name.c * src/wydawca.h (wy_set_cur_thread_name): New proto. * src/net.c: Set thread names. * src/triplet.c: Likewise. * src/wydawca.c: Likewise.
2020-04-24Use AX_PTHREAD to detect posix thread supportSergey Poznyakoff5
2020-04-24Fix the gpg2 version requirement. Document changes to the testsuiteSergey Poznyakoff4
2020-04-23Add test caseSergey Poznyakoff1
2020-04-23Remove a left-over global variableSergey Poznyakoff6
2020-04-23Rewrite the testsuite.Sergey Poznyakoff53
Use gpg2 to create the keys, and sign files as necessary. * NEWS: Update. * doc/wydawca.texi: Update. * configure.ac: Detect the presence of GPG version 2. * src/cmdline.opt (--single-process): Remove. * src/config.c: Remove the "single-process" statement. * src/triplet.c (triplet_enqueue): Fix eventual NULL dereference. (wy_triplet_wait): Update the WY_STAT_INCOMPLETE_TRIPLETS counter. * src/wydawca.c (single_process_option): Remove global. * src/wydawca.h (single_process): Remove global. * tests/.gitignore: Update. * tests/Makefile.am: Update. * tests/atlocal.in: Rewrite. * tests/chargen.c: New auxiliary tool * tests/dist/file: Remove. * tests/dist/file.directive.asc: Remove. * tests/dist/file.sig: Remove. * tests/dist/file12f.directive.asc: Remove. * tests/dist/file12t.directive.asc: Reemove. * tests/dist/rmsymlink.directive.asc: Remove. * tests/dist/symlink.directive.asc: Remove. * tests/etc/notify.cfin: Rename to tests/etc/notify.cf * tests/etc/Makefile.am: Update. * tests/etc/dbgen.sh: New file. * tests/etc/keygen.sh: New file. * tests/etc/project: New file. * tests/etc/pubring.asc: Remove. * tests/etc/secring.asc: Remove. * tests/etc/wydawca.cfin: Remove. * tests/*.at: Rewrite.
2020-04-23Fix make distcheckSergey Poznyakoff2
* tests/unp01.at: Fix GNUPGHOME to point to the directory in the build tree. * tests/etc/Makefile.am: Augment DISTCLEANFILES
2020-04-22Avoid double-checking of the directive files. Fix triplet expiration.Sergey Poznyakoff7
* doc/wydawca.texi: Document the default value for file-sweep-time. * src/config.c (parse_single_statmask): Remove the invert pointer. (parse_statmask): Reflect the above. (config_finish): Ensure safe value for file_sweep_time. * src/triplet.c (register_file): Initialize the directive_verified member. (triplet_expired_p): No special handling for ttl==0. * src/verify.c (verify_directive_file): Cache the result in trp->directive_verified. * src/wydawca.c (file_sweep_time): Initialize to DEFAULT_FILE_SWEEP_TIME. * src/wydawca.h (DEFAULT_FILE_SWEEP_TIME): New constant. (DIRECTIVE_UNCHECKED,DIRECTIVE_GOOD) (DIRECTIVE_BAD): New constants. (struct wy_triplet): New member: directive_verified. * tests/cwdrepl.c: Fix a prematurely closed comment.
2020-04-20Add missing copyright statementsSergey Poznyakoff3
2020-04-20Add tests for legacy upload notification interfaceSergey Poznyakoff13
* src/net.c (handle_connection): Ignore user/comment. * tests/.gitignore: Add wyclt. * tests/wyclt.c: New file. * tests/Makefile.am (TESTSUITE_AT): Add new tests. * tests/unp-idle.at: New test. * tests/unp00.at: New test. * tests/unp01.at: New test. * tests/testsuite.at (AT_WYDAWCA_DAEMON): Change arguments. (AT_WYDAWCA_INOTIFY_DAEMON): New macro. Include new tests.
2020-04-20Minor fixes in the docsSergey Poznyakoff1
2020-04-20Rewrite the timer support.Sergey Poznyakoff9
Timers are kept in a thread-specific array variable. Spool timers are referenced in expansion strings by their fully qualified names, e.g. ${timer:spool:release:system}. * NEWS: Update. * doc/wydawca.texi: Document new syntax for the spool timer variables. * src/directive.c * src/wydawca.h (struct spool): New member: timer_id. * tests/upload-dry.at: Filter out certain messages, that can appear * src/process.c (spool_count): New global. (register_spool): Build spool list in the same order as listed in the configuration file. (spool_timer_id): New function. * src/timer.c: Rewrite using array of struct timer_slot, instead of the hash table. * src/triplet.c (try_timer_var): Rewrite. * src/wydawca.c (stderr_printer): Protect stderr by a mutex. in unpredictable order.
2020-04-18Modify pidfile functionsSergey Poznyakoff3
2020-04-18Fix statistics gathering. Display stats at the end of the run in cron mode.Sergey Poznyakoff8
* src/directive.c (process_directives): Raise the 'destroying' flag prior to removing the directive file, in order to avoid the removal being caught by the watcher thread. * src/timer.c (wydawca_stats_export, wydawca_stats_update): New functions. * src/triplet.c (triplet_list_lock,triplet_list_unlock): Check for NULL pointer. (remove_triplet_unlocked): Set the 'destroying' flag. (triplet_remove_file): Do nothing if the 'destroying' flag is set. (wy_thr_triplet): Call logstats only in the daemon mode. Otherwise call wydawca_stats_update. * src/wydawca.c (wy_main): Call wydawca_stats_export at the beginning. Call logstats before exiting, unless in daemon mode. * src/wydawca.h (struct wy_triplet): New member: destroying. (wydawca_stats_export,wydawca_stats_update): New protos. * tests/check-fail.at: Update. * tests/check-notify.at: Update. * tests/upl12f.at: Update.
2020-04-18Remove spool lockingSergey Poznyakoff11
* NEWS: Document changes. * doc/wydawca.texi: Document changes. * src/Makefile.am: Remove lock.c * src/lock.c: Delete. * src/config.c: Remove the locking statement. * tests/etc/wydawca.cfin: Likewise. * src/wydawca.c: Check pidfile before startup, unless in dry-run mode. * src/wydawca.h: Remove prototypes of locking functions * src/process.c (scan_spool_unlocked): Merge into scan_spool. * src/timer.c (struct timer_slot): Use reference counter instead of timer state
2020-04-18Minor fixes.Sergey Poznyakoff2
* src/timer.c (struct timer_slot): New member: state. (timer_start,timer_stop): Keep track of the timer state. * src/triplet.c: Remove unneeded function.
2020-04-17Improve configure.acSergey Poznyakoff1
* configure.ac: Check for librt and gpgme_set_offline.
2020-04-17Update the docsSergey Poznyakoff3
2020-04-17Limit number of simultaneous tcpmux connections. Impose idle connection ↵Sergey Poznyakoff5
timeouts. * src/config.c: New statements: max-connections and idle-timeout * src/net.c (max_connections,idle_timeout): New globals. (handle_connection): Take single FILE* as argument. (connection_start,connection_stop): New functions. (wy_thr_tcpmux,wy_thr_connection_watcher): New functions. (wy_thr_listen): Start wy_thr_connection_watcher thread if tcpmux listener is enabled. Start a separate thread to process each incoming connection. * src/wydawca.h (max_connections,idle_timeout): New externs. (wy_thr_tcpmux,wy_thr_connection_watcher): New protos.
2020-04-17Minor changesSergey Poznyakoff2
2020-04-16Rewrite as a multi-threaded program.Sergey Poznyakoff20
* NEWS: Update. * configure.ac: Version 3.1.95 * src/Makefile.am (wydawca_SOURCES): Remove job.c (LDADD): Link with libpthread. * src/job.c: Remove. * src/config.c: Remove the wakeup-interval statement. * src/net.c (wydawca_listener): Rewrite as a thread function wy_thr_listen. * src/timer.c (timer_get): Use thread-specific storage. (timer_get_count): Remove. * src/triplet.c: Rewrite using multi-thread model. * src/wydawca.c: Likewise. * src/wydawca.h: Update. * tests/check-fail.at: Update expected output. * tests/check-notify.at: Likewise. * tests/inotify-ok.at: Likewise. * tests/inotify-rmsymlink.at: Likewise. * tests/inotify-symlink.at: Likewise. * tests/inotify-unatt00.at: Likewise. * tests/inotify-unatt01.at: Likewise. * tests/upl12f.at: Likewise. * tests/upload-dry.at: Likewise. `
2020-04-15Use job scheduler when processing all spools.alpha-3.1.90Sergey Poznyakoff8
* src/job.c (job_printable): Improve job identification. (wydawca_scanner): Don't log statistics in single process mode. (job_queue_wait): New function. * src/net.c (wydawca_listener): Move the call to job_init to main. * src/process.c (scan_spool_unlocked): Enqueue the triplet. Don't call spool_commit_triplets * src/triplet.c (remove_triplet): Omit checks in dry-run mode. (triplet_processor,spool_commit_triplets) (count_collected_triplets): Remove. * src/wydawca.c (main): Call job_queue_wait if not daemon. * tests/inotify-unatt01.at: Update expected output. * tests/upload-dry.at: Likewise.
2020-04-15Update the manualSergey Poznyakoff1
2020-04-15Rewrite GPGME supportSergey Poznyakoff10
* src/Makefile.am: Remove pushd.c * src/gpg.c: Rewrite. * src/pushd.c: Remove. * src/triplet.c (remove_triplet): Release the gpgme context. * src/wydawca.h: Include gpgme.h (struct wy_triplet): New member: gpgme_ctx; (triplet_gpgme_ctx_release): New proto. * tests/pushck.c: Remove. * tests/pushdir.at: Remove. * tests/Makefile.am: Remove pushck * tests/.gitignore: Likewise. * tests/testsuite.at: Remove pushdir test.
2020-04-15Replace UPDATE_STATS macro with inline function increase_stat_counterSergey Poznyakoff7
2020-04-15When processing spool requests, handle each completed triplet in spool.Sergey Poznyakoff16
Previously wydawca used to take optional second parameter - a list of submitter UIDs. If given this list, it would process only those triplets that had been uploaded by one of these UIDs. This proved to be unnecessary. The same change is applied to the notification protocol. Only the spool name is required. The second word in the request (user name) is now silently ignored. * src/wydawca.h (struct wy_triplet): Remove const qualifier from the spool member. Remove jq_prev and jq_next pointers. (struct job_spool_id): Remove. (job_schedule_spool): Change signature. * src/job.c (struct job): Replace job_spool with a pointer to spool. (triplet_jq_unlink,jq_clear): Remove. (job_printable): Change formatting of JOB_SPOOL jobs. (job_locate): Update. (wydawca_scanner): Update. * src/net.c (handle_connection): Ignore second word in the request. * src/process.c (spool_add_new_file): Change signature. UIDs are no longer needed. Return a pointer to the affected triplet. (scan_spool): Change signature. * src/triplet.c (register_file): Return pointer to the affected triplet. (check_triplet_state): Remove the noauth parameter. (triplet_enqueue): New function. (triplet_commit): New function. (spool_commit_triplets): Remove the tplist parameter. Process all complete triplets in the spool. (link_processable_triplets): Remove. * src/verify.c (verify_directive_file): Remove the noauth parameter. * src/watcher.c (process_event): Enqueue created triplets. * src/wydawca.c (main): Don't take any UID arguments. * tests/inotify-ok.at: The version statement is printed once. Update the expected output. * tests/inotify-rmsymlink.at: Likewise. * tests/inotify-symlink.at: Likewise. * tests/inotify-unatt01.at: Likewise.
2020-04-14testsuite: pass exactly 1 argument to the readlink utilitySergey Poznyakoff3
2020-04-14testsuite: test whether unattended uploads are processed at daemon start upSergey Poznyakoff7
2020-04-13Rewrite diskio function so as not to assume any particular cwdSergey Poznyakoff20
* src/backup.c (get_backup_suffix) (find_backup_file_name): Get directory descriptor as first argument. * src/diskio.c: Rewrite using *at() functions. * src/gpg.c (rmdir_r): Silently skip ENOENT. (verify_detached_signature): push_dir/pop_dir. * src/triplet.c: Don't change to spool->source_dir. Underlying functions don't assume any specific cwd. * src/wydawca.h (find_backup_file_name) (create_hierarchy): Change signatures. All uses changed. * tests/Makefile.am: Add new tests. * tests/testsuite.at: Likewise. (AT_WYDAWCA_DAEMON): Take additional argument. * tests/inotify-rmsymlink.at: New test. * tests/inotify-symlink.at: New test. * tests/rmsymlink00.at: New test. * tests/symlink00.at: New test. * tests/symlink01.at: New test. * tests/dist/rmsymlink.directive.asc: New directive file. * tests/dist/symlink.directive.asc: New directive file.
2020-04-13Fix triplet processing in daemon mode.Sergey Poznyakoff33
* src/watcher.c (fopenat_ro): New function. * src/verify.c (fill_project_name): Use fopenat_ro. * src/gpg.c (recursive_rmdir): return silently if the directory does not exist. (verify_directive_signature): Use fopenat_ro. * src/job.c: Change state definitions from bitmask to enum. * src/triplet.c (triplet_processor): A pointer to the spool is passed in proc_data. Process only triplets with this spool. (triplet_processor): Modify accordingly. Facilitate writing tests for daemon mode. * src/wydawca.c (cron_option,daemon_mode): Remove. (wy_mode, wy_mode_option): New variables. (main): Use wy_mode to determine operating mode. * src/wydawca.h (daemon_mode): Remove extern. (wy_mode): New extern. * src/cmdline.opt: New option --daemon. * src/config.c: Change handling of the "daemon" statement. * src/net.c: Test hook: if WYDAWCA_NOTIFY_PARENT=1 send SIGUSR1 to parent after setting up the listener. * doc/wydawca.texi: Describe the --daemon option. Testsuite: 1. Place source and destination directories in the working directory for each particular test. Thus the spool cleanup function becomes unnecessary. 2. Add tests for wydawca daemon mode (inotify). * configure.ac (WY_INOTIFY): New substitution variable for use in testsuite. * tests/wyasync.c: New source. Auxiliary program to test wydawca daemon mode. * tests/wyinit.c: New source. Extract source and destination directory names from the wydawca configuration file and create missing directories. * tests/Makefile.am: Build wyasync and wyinit. (TESTSUITE_AT): Add inotify-ok.at * tests/atlocal.in (WY_INOTIFY): New variable. (wydawca_init_testdirs): Remove. (wydawca_clean_testdirs): Remove. (wydawca_config): Call wyinit to initialize spools and destinatiuon directories. (wydawca_upload): Change source directory. * tests/etc/mailstats.cfin: Put mail.out in CWD. * tests/etc/notify.cfin: Likewise. * tests/etc/wydawca.cfin: Add pidfile statement in case wydawca is run as daemon. Place all source and destination directories in CWD. * tests/testsuite.at (AT_WYDAWCA_DAEMON): New macro. * tests/inotify-ok.at: New test. * tests/check-fail.at: Update. * tests/check-notify.at: Update. * tests/check-ok.at: Update. * tests/mailstats.at: Update. * tests/notify-upl.at: Update. * tests/upl11.at: Update. * tests/upl12f.at: Update. * tests/upl12t.at: Update. * tests/upload-dry.at: Update. * tests/upload.at: Update.
2020-04-10Change indentation to improve readability.Sergey Poznyakoff37
2020-04-10Avoid unnecessary directory changes.Sergey Poznyakoff8
* src/config.c (cb_spool): Initialize spool->source_fd. * src/process.c (spool_add_new_file): Remove. (spool_cwd_add_new_file): Rename to spool_add_new_file. Use fstatat instead of stat. (scan_spool_unlocked): Don't change to spool->source_dir. * src/triplet.c (remove_triplet): Don't change cwd. Use fstatat to check for existence and unlinkat to remove the file. (spool_commit_triplets): Change to the spool source directory here. * src/wydawca.h: Define O_SEARCH to 0 if not defined. (struct spool): New member source_fd. * tests/check-fail.at: Update expected output. * tests/check-notify.at: Likewise. * tests/upl12f.at: Likewise.
2020-04-10Scan configured spools before entering daemon loop.Sergey Poznyakoff6
This ensures that eventual submissions that may have occurred during daemon restarts will be duly processed. * src/wydawca.h (fake_spool, inotify_spool): Remove pseudo-spools. (dictionaries_close): New proto. (job_schedule): New proto. (job_schedule_all, job_schedule_triplet) (job_schedule_spool): New inline functions. * src/job.c (struct job): New member "type". Move spool and uid to union v. Each job can have either triplet list or (spool,uid) pair, but not both. (fake_spool, inotify_spool): Remove. New member "printable". (job_locate): Rewrite. (wydawca_scanner): Rewrite. (job_destroy): New function. (schedule_job): Remove. (job_schedule): New function. (job_printable): New function. (all functions): Use job_printable to identify jobs in diagnostic messages. * src/net.c (handle_connection): Use job_schedule_spool or job_schedule_all depending on the request. * src/process.c (dictionaries_close): New function. (scan_all_spools): Don't close dictionaries. * src/watcher.c (process_event): Initiate the job using job_schedule_triplet(). * src/wydawca.c (main): Don't allow extra arguments in daemon mode. Call scan_all_spools before entering wydawca_daemon to register existing files. Close dictionaries before closing modules.
2020-04-09Minor documentation changeSergey Poznyakoff1
2020-04-09Change configuration file name to wydawca.confSergey Poznyakoff20
Configuration files in the testsuite use the ".cf" suffix.
2020-04-09Bugfixes.Sergey Poznyakoff1
* src/triplet.c (triplet_list_unlink): Avoid accidentally clearing the list. (hash_triplet_compare): Fix return value. (register_file): Don't attempt to unlink a newly installed file. (remove_triplet): Change to the spool source directory.
2020-03-13Minor changeSergey Poznyakoff1
2020-03-13Version 3.1release-3.1Sergey Poznyakoff76
2020-03-13Fix .gitignore filesSergey Poznyakoff3
2020-03-13BugfixesSergey Poznyakoff3
* Makefile.am (ACLOCAL_AMFLAGS): Remove am * am/mailutils.m4: Remove. * src/gpg.c (recursive_rmdir): Don't bail out on ENOENT.
2020-03-13Fix the push/pop dir test.Sergey Poznyakoff3
The test failed on machines where $(pwd -P) != $(pwd -L). * tests/Makefile.am: Add cwdrepl.c * tests/cwdrepl.c: New file. * tests/pushdir.at: Use cwdrepl to filter the output.
2019-07-24Use wordsplit to expand variables in strings.Sergey Poznyakoff18
* NEWS: Raise version number. * configure.ac: Link wydawca.h to include/wydawca Raise version number * doc/wydawca.texi: Document changes. * etc/wydawca.rc: Fix the syntax. * grecs: Upgrace. * include/wydawca/wydawca.h (wy_vlog): New proto. (wy_log, wy_dbg): Format is const char *. (wy_triplet_t): Change typedef. (wy_metadef): Remove struct. (wy_vardef): New struct. (wy_triplet_expand_param) (wy_expand_copy): New protos. * modules/logstat/mod_logstat.c: Update. * modules/mailutils/Makefile.am (AM_CPPFLAGS): Modify, * modules/mailutils/mod_mailutils.c: Update. * src/Makefile.am (wydawca_SOURCES): Remove meta.c * src/dictionary.c: Fix comment. * src/directive.c (directive_get_value): Triplet pointer is const. * src/meta.c: Remove. * src/timer.c: (timer_fill_meta) (timer_free_meta): Remove. * src/triplet.c: Use wordsplit for expansions. * src/wydawca.c (wy_vlog): New function. (wy_stat_expansion): New function. * src/wydawca.h: Remove metadef protos. * tests/etc/notify.rcin: Update variable reference syntax.
2019-07-22Update copyright years in version output (better late, than never)Sergey Poznyakoff1
2019-07-22Verion 3.0release-3.0Sergey Poznyakoff5
2019-07-22Install default pp-setup file from grecsSergey Poznyakoff3
2019-07-22Document directory setup statements.Sergey Poznyakoff1
See commits f338532e, dd9d9c12, and af538cc2.

Return to:

Send suggestions and report system problems to the System administrator.