aboutsummaryrefslogtreecommitdiff
path: root/src/falloc.c
AgeCommit message (Collapse)AuthorFiles
2023-01-22Update copyright yearsSergey Poznyakoff1
2022-01-06Speed up flushing the bucket cache on diskSergey Poznyakoff1
The implementation of _gdbm_cache_flush becomes prohibitively inefficient during extensive updates of large databases. The bug was reported at https://github.com/Perl/perl5/issues/19306. To fix it, make sure that all changed cache entries are placed at the head of the cache_mru list, forming a contiguous sequence. This way a potentially long iteration over all cache entries can be cut off at the first entry with ca_changed == FALSE. This commit also gets rid of several superfluous fields in struct gdbm_file_info: - cache_entry Not needed, because the most recently used cache entry (cache_mru) is always the current one. - bucket_changed dbf->cache_mru->ca_changed reflects the status of the current bucket. - second_changed Not needed because _gdbm_cache_flush, which flushes all changed buckets, is now invoked unconditionally by _gdbm_end_update (and also whenever dbf->cache_mru changes). * src/gdbmdefs.h (struct gdbm_file_info): Remove cache_entry. The current cache entry is cache_mru. Remove bucket_changed, and second_changed. All uses changed. * src/proto.h (_gdbm_current_bucket_changed): New inline function. * src/bucket.c (_gdbm_cache_flush): Assume all changed elements form a contiguous sequence beginning with dbf->cache_mru. (set_cache_entry): Remove. All callers changed. (lru_link_elem,lru_unlink_elem): Update dbf->bucket as necessary. (cache_lookup): If the obtained bucket is not changed and is going to become current, flush all changed cache elements. * src/update.c (_gdbm_end_update): Call _gdbm_cache_flush unconditionally. * src/findkey.c: Use dbf->cache_mru instead of the removed dbf->cache_entry. * src/gdbmseq.c: Likewise. * tools/gdbmshell.c (_gdbm_print_bucket_cache): Likewise. * src/falloc.c: Use _gdbm_current_bucket_changed to mark the current bucket as changed. * src/gdbmstore.c: Likewise. * src/gdbmdelete.c: Likewise. Use _gdbm_current_bucket_changed. * tests/gtcacheopt.c: Fix typo. * tests/gtload.c: New option: -cachesize
2022-01-02Update copyright yearsSergey Poznyakoff1
2021-11-18Fix typosSergey Poznyakoff1
2021-09-16Fix computation of dbf->avail->count after pushing new avail block.Sergey Poznyakoff1
This is a very long-standing bug that caused a loss of an avail entry if the original avail table had odd number of entries. * src/falloc.c (push_avail_block): Fix computation of dbf->avail->count.
2021-07-28Prepare structs gdbm_file_header and gdbm_file_info for changes.Sergey Poznyakoff1
* src/gdbmconst.h (GDBM_NUMSYNC_MAGIC32) (GDBM_NUMSYNC_MAGIC64) (GDBM_NUMSYNC_MAGIC32_SWAP) (GDBM_NUMSYNC_MAGIC64_SWAP): New constants. * src/gdbmdefs.h (gdbm_file_header): Move avail to a union in anticipation of further changes. (gdbm_file_info) <avail, avail_size>: New members. (GDBM_HEADER_AVAIL_SIZE): Redefine. * src/gdbmopen.c (GDBM_NUMSYNC_MAGIC): New define. (validate_header_std) (validate_header_numsync): New functions. (validate_header): Use one of these depending on the magic number. (gdbm_fd_open): Initialize new members of gdbm_file_info. * src/avail.c: Use the GDBM_FILE avail pointer. * src/falloc.c: Likewise. * src/gdbmtool.c: Likewise.
2021-03-16Improve reading and validation of available blocks (avail_block).Sergey Poznyakoff1
This fixes remaining failures in http://puszcza.gnu.org.ua/bugs/?501. * src/falloc.c (_gdbm_avail_block_read): New function. (pop_avail_block): Use _gdbm_avail_block_read. * src/gdbmdefs.h: Fix typo in the comment. * src/gdbmopen.c (gdbm_avail_block_validate): Take size of the avail block in bytes as the third argument. All uses changed. (PARTIAL_HEADER_SIZE) (HEADER_AVAIL_SIZE): New macros. (_gdbm_validate_header): Validate the avail block as well. (validate_header): Don't validate the avail block fields here. (gdbm_fd_open): Read partial header up to the avail field, then read entire avail block using _gdbm_avail_block_read. * src/gdbmtool.c (_gdbm_print_avail_list): Fix avail block calculation (one extra entry was assumed). Use _gdbm_avail_block_read to obtain and validate block. * src/proto.h (_gdbm_avail_block_read): New proto. (gdbm_avail_block_validate): Change signature.
2021-01-02Update copyright yearsSergey Poznyakoff1
2020-12-23Update copyright yearsSergey Poznyakoff1
2019-04-08Update copyright yearsSergey Poznyakoff1
2018-08-31Various bugfixes.Sergey Poznyakoff1
* compat/dbmopen.c (ndbm_open_dir_file0): Ignore ENOENT. * src/falloc.c (push_avail_block): Free temporary storage no matter what return status. * src/gdbm.h.in (GDBM_FILE_TRUNCATE_ERROR): New error code. * src/gdbmdump.c (_gdbm_dump_ascii): Initialize rc. * src/gdbmerrno.c: Handle new error.code * src/gdbmload.c (gdbm_load_bdb_dump): Initialize rc * src/gdbmopen.c (_gdbm_ftruncate): New function. (gdbm_fd_open): Use _gdbm_ftruncate. Check its return. * src/gdbmseq.c (gdbm_firstkey): Initialize dsize * src/gdbmtool.c (command_generator): Check if cmd is NULL. (shouldn't happen, but anyways). * src/mmap.c (_gdbm_mapped_lseek): Check for vailidity of the 'whence' parameter. * src/systems.h (TRUNCATE): Remove macro. * src/util.c (vgetyn): Remove unnecessary assignment.
2018-06-27Coalesce mode: take into account both left- and right-adjacent slots.Sergey Poznyakoff1
* src/falloc.c (avail_lookup): Remove the start parameter. (avail_move): Take pointer to the count of entries in av_table. Update the pointed to value after performing the move. All uses changed. (_gdbm_put_av_elem): Rewrite the "can_merge" loop.
2018-06-25Optimize operations on avail lists.Sergey Poznyakoff1
Use binary search for look-ups. * src/falloc.c (avail_lookup, avail_move): New functions. (get_elem): Use avail_lookup to search. (_gdbm_put_av_elem): Use avail_lookup and avail_move to handle the avail table. In coalesce mode, store the index of the greater than or equal entry to avoid extra lookup in case no suitable entry is found.
2018-06-23Bugfixes. Consistently use the coalesce_blocks flag.Sergey Poznyakoff1
* src/bucket.c (_gdbm_split_bucket): Take into account coalesce_blocks * src/falloc.c (_gdbm_put_av_elem): Change to void return. All uses changed. * src/proto.h: Likewise.
2018-06-23Maintain correct ordering of the avail table after coalescing.Sergey Poznyakoff1
* src/falloc.c (_gdbm_put_av_elem): Float the merged entry to its proper position to preserve the sorting order. * THANKS: Thanks Adam Sampson
2018-06-16Version 1.5v1.15Sergey Poznyakoff1
* configure.ac: Version 1.5 * NEWS: Update. * src/Makefile.am (VI_CURRENT): Increment to 6 * src/bucket.c (_gdbm_get_bucket): Fix the upper limit for bucket_bits. * src/falloc.c (push_avail_block): Fill the allocated block with 0s.
2018-05-30Namespace cleanupSergey Poznyakoff1
Rename: __read to gdbm_file_read __write to gdbm_file_write __lseek to gdbm_file_seek __fsync to gdbm_file_sync
2018-05-30Drop debugging hooksSergey Poznyakoff1
The hooks were introduced as a temporary tool in de7834e9. They did their job and are not necessary any more.
2018-05-27Additional validation of avail_table.Sergey Poznyakoff1
Verify that avail_table is sorted by size and that each element's size falls within allowed range. * src/bucket.c (gdbm_get_bucket): Fix bucket validation. Validate bucket_avail. (_gdbm_split_bucket): Check return from _gdbm_free. * src/falloc.c (adjust_bucket_avail,_gdbm_free): Return error code. All uses updated. (pop_avail_block): Fix eventual memory leak. Use gdbm_avail_block_validate. * src/gdbmdefs.h (gdbm_avail_table_valid_p): Change signature. * src/gdbmopen.c (gdbm_avail_table_valid_p): Traverse the array verifying address and size of each element. (gdbm_avail_block_validate) (gdbm_bucket_avail_table_validate): New functions. (validate_header): Remove call to gdbm_avail_block_valid_p. Avail_block is validated later, after it's been loaded. Bail out if header->next_block does not equal the file size. (gdbm_fd_open): Validate avail_block. * src/gdbmstore.c (_gdbm_store): Check return from _gdbm_free. Avoid endless loop in case of inconsistent h_table. * src/gdbmtool.c (_gdbm_print_avail_list): Use gdbm_avail_block_validate. * src/proto.h: Update. * tests/gtload.c: Improve error diagnostics.
2018-05-24More input checking fixesSergey Poznyakoff1
* src/gdbmdefs.h (gdbm_avail_block_valid_p): Minimal capacity of an avail_block is 1 (one entry is stored in the block itself). * src/gdbmopen.c (gdbm_avail_table_valid_p): New function. (validate_header): Fix verification of dir_size and dir_bits. Verify avail.size. * src/falloc.c (pop_avail_block): Use gdbm_avail_table_valid_p * src/gdbmtool.c (_gdbm_print_avail_list): Likewise.
2018-05-24Fix the use of return code from _gdbm_full_(read|write) functionsSergey Poznyakoff1
2018-05-24error checking: check for avail_block consistency before using itSergey Poznyakoff1
* src/gdbmdefs.h (gdbm_avail_block_valid_p): New function. * src/gdbm.h.in (GDBM_BAD_AVAIL): New error code. * src/gdbmerrno.c: Support new error code. * src/falloc.c (pop_avail_block): Validate the avail_block * src/gdbmopen.c (validate_header): Validate the avail_block. * src/gdbmtool.c (_gdbm_avail_list_size) (_gdbm_print_avail_list): Validate the avail_block.
2018-01-01Happy GNU YearSergey Poznyakoff1
2017-01-02Happy GNU YearSergey Poznyakoff1
2016-07-26Fix remaining uses of gdbm_set_errno function.Sergey Poznyakoff1
Use the GDBM_SET_ERRNO and GDBM_SET_ERRNO2 macros to make sure the error gets reported in debug output. * src/fullio.c (_gdbm_full_read) (_gdbm_full_write): Return -1 and set gdbm_errno on error. * src/bucket.c: Use GDBM_SET_ERRNO(2?) or GDBM_DEBUG where necessary. * src/falloc.c: Likewise. * src/findkey.c: Likewise. * src/gdbmdefs.h: Likewise. * src/gdbmopen.c: Likewise. * src/gdbmstore.c: Likewise. * src/mmap.c: Likewise. * src/recover.c: Likewise. * src/update.c: Likewise.
2016-07-25Improve debugging and error tracing.Sergey Poznyakoff1
* src/gdbmdefs.h (GDBM_SET_ERRNO) (GDBM_SET_ERRNO2): New macros. * src/base64.c: Use new macros to set error state. * src/bucket.c: Likewise. * src/falloc.c: Likewise. * src/findkey.c: Likewise. * src/gdbm_load.c: Likewise. * src/gdbmdelete.c: Likewise. * src/gdbmdump.c: Likewise. * src/gdbmexp.c: Likewise. * src/gdbmfetch.c: Likewise. * src/gdbmimp.c: Likewise. * src/gdbmload.c: Likewise. * src/gdbmopen.c: Likewise. * src/gdbmseq.c: Likewise. * src/gdbmsetopt.c: Likewise. * src/gdbmstore.c: Likewise. * src/gdbmtool.c: Fix preprocessor conditional.
2016-07-20Introduce debug hooks.Sergey Poznyakoff1
* configure.ac: New option --enable-debug Print feature summary at the end of the run. * src/debug.c: New file. * src/Makefile.am [GDBM_COND_DEBUG_ENABLE]: Build debug.o Define GDBM_DEBUG_ENABLE. * src/gdbmdefs.h [GDBM_DEBUG_ENABLE] (_gdbm_debug_hook_install) (_gdbm_debug_hook_remove,_gdbm_debug_hook_check) (_gdbm_debug_hook_val): New protos. (GDBM_DEBUG_HOOK, GDBM_DEBUG_OVERRIDE) (GDBM_DEBUG_ALLOC): New defines. * src/gdbm.h.in (GDBM_RCVR_FORCE): New flag. * src/recover.c (gdbm_recover): Check database before attempting recovery, unless GDBM_RCVR_FORCE flag is set. * doc/gdbm.texi: Document GDBM_RCVR_FORCE * src/gdbmreorg.c (gdbm_reorganize): Use GDBM_RCVR_FORCE. * src/gdbmtool.c (main): Always allocate file_name. * src/bucket.c: Put GDBM_DEBUG_OVERRIDE and GDBM_DEBUG_ALLOC in critical places. * src/falloc.c: Likewise. * src/findkey.c: Likewise. * src/gdbmopen.c: Likewise. * src/gdbmstore.c: Likewise. * src/update.c: Likewise. * tests/Makefile.am [GDBM_COND_DEBUG_ENABLE]: Define GDBM_DEBUG_ENABLE. * tests/gtload.c: New options -hook, -recover, -verbose, -backup, -max-failures, -max-failed-keys, and -max-failed-buckets. Attempt recovery after errors.
2016-07-09Don't bail out on fatal errors, unless the user defines the fatal_err functionSergey Poznyakoff1
* src/bucket.c (_gdbm_get_bucket) (_gdbm_split_bucket, _gdbm_write_bucket): Return error code. All callers updated. * src/proto.h (_gdbm_get_bucket) (_gdbm_split_bucket, _gdbm_write_bucket): Update declarations * src/falloc.c (push_avail_block) (pop_avail_block): Return error code. All callers updated. * src/update.c (_gdbm_fatal): Exit only if the user defined fatal_err function. * src/gdbmerrno.c (gdbm_needs_recovery): New function. * src/gdbm.h.in (gdbm_needs_recovery): New proto. * compat/dbminit.c: Set gdbm_errno on fatal errors. * compat/dbmopen.c: Likewise. * src/findkey.c: Likewise. * src/gdbm_load.c: Likewise. * src/gdbmcount.c: Likewise. * src/gdbmdefs.h: Likewise. * src/gdbmdelete.c: Likewise. * src/gdbmdump.c: Likewise. * src/gdbmexists.c: Likewise. * src/gdbmexp.c: Likewise. * src/gdbmfetch.c: Likewise. * src/gdbmimp.c: Likewise. * src/gdbmload.c: Likewise. * src/gdbmopen.c: Likewise. * src/gdbmreorg.c: Likewise. * src/gdbmseq.c: Likewise. * src/gdbmsetopt.c: Likewise. * src/gdbmstore.c: Likewise. * src/gdbmsync.c: Likewise. * src/mmap.c: Likewise.
2013-12-25Version 1.11Sergey Poznyakoff1
2011-11-11Use SEEK_SET instead of L_SET.Sergey Poznyakoff1
2011-10-05Fix bug #150.Sergey Poznyakoff1
Tolerate I/O operations returning less bytes than expected. Retry I/O if possible. * src/fullio.c: New file. * src/Makefile.am (libgdbm_la_SOURCES): Add fullio.c * src/proto.h (_gdbm_full_read) (_gdbm_full_write): New protos. * src/gdbmerrno.c (gdbm_errlist): Add entry for GDBM_FILE_EOF. * src/bucket.c: Use _gdbm_full_{read|write}. * src/falloc.c: Likewise. * src/findkey.c: Likewise. * src/gdbmopen.c: Likewise. * src/gdbmstore.c: Likewise. * src/testgdbm.c: Likewise. * src/update.c: Likewise.
2011-08-16Internationalization.Sergey Poznyakoff1
* Makefile.am (SUBDIRS): Add po. (EXTRA_DIST): Add build-aux/config.rpath. * configure.ac (AC_CONFIG_FILES): Add po/Makefile.in. * bootstrap: New file. * src/Makefile.am (AM_CPPFLAGS): Define LOCALEDIR (noinst_HEADERS): Add gettext.h (LIBADD): New variable. * src/gettext.h: New file. * po/.cvsignore: New file. * po/Makevars: New file. * po/POTFILES.in: New file. * src/gdbmdefs.h: Define DEFAULT_TEXT_DOMAIN, _, N_ Include gettext.h * src/bucket.c: Add NLS markers. * src/falloc.c: Likewise. * src/findkey.c: Likewise. * src/gdbmerrno.c: Likewise. * src/gdbmfetch.c: Likewise. * src/gdbmseq.c: Likewise. * src/gdbmstore.c: Likewise. * src/update.c: Likewise. * src/testgdbm.c: Add NLS markers. (main): Initialize I18N.
2011-08-03Update copyright headers.Sergey Poznyakoff1
2011-08-01Use GDBM_FILE instead of gdbm_file_info *.Sergey Poznyakoff1
2008-11-21Initial revisionJason Downs1

Return to:

Send suggestions and report system problems to the System administrator.