aboutsummaryrefslogtreecommitdiff
path: root/src/gdbmdelete.c
AgeCommit message (Collapse)AuthorFiles
2022-01-06Speed up flushing the bucket cache on diskSergey Poznyakoff1
The implementation of _gdbm_cache_flush becomes prohibitively inefficient during extensive updates of large databases. The bug was reported at https://github.com/Perl/perl5/issues/19306. To fix it, make sure that all changed cache entries are placed at the head of the cache_mru list, forming a contiguous sequence. This way a potentially long iteration over all cache entries can be cut off at the first entry with ca_changed == FALSE. This commit also gets rid of several superfluous fields in struct gdbm_file_info: - cache_entry Not needed, because the most recently used cache entry (cache_mru) is always the current one. - bucket_changed dbf->cache_mru->ca_changed reflects the status of the current bucket. - second_changed Not needed because _gdbm_cache_flush, which flushes all changed buckets, is now invoked unconditionally by _gdbm_end_update (and also whenever dbf->cache_mru changes). * src/gdbmdefs.h (struct gdbm_file_info): Remove cache_entry. The current cache entry is cache_mru. Remove bucket_changed, and second_changed. All uses changed. * src/proto.h (_gdbm_current_bucket_changed): New inline function. * src/bucket.c (_gdbm_cache_flush): Assume all changed elements form a contiguous sequence beginning with dbf->cache_mru. (set_cache_entry): Remove. All callers changed. (lru_link_elem,lru_unlink_elem): Update dbf->bucket as necessary. (cache_lookup): If the obtained bucket is not changed and is going to become current, flush all changed cache elements. * src/update.c (_gdbm_end_update): Call _gdbm_cache_flush unconditionally. * src/findkey.c: Use dbf->cache_mru instead of the removed dbf->cache_entry. * src/gdbmseq.c: Likewise. * tools/gdbmshell.c (_gdbm_print_bucket_cache): Likewise. * src/falloc.c: Use _gdbm_current_bucket_changed to mark the current bucket as changed. * src/gdbmstore.c: Likewise. * src/gdbmdelete.c: Likewise. Use _gdbm_current_bucket_changed. * tests/gtcacheopt.c: Fix typo. * tests/gtload.c: New option: -cachesize
2022-01-02Update copyright yearsSergey Poznyakoff1
2021-01-02Update copyright yearsSergey Poznyakoff1
2020-12-23Update copyright yearsSergey Poznyakoff1
2019-04-08Update copyright yearsSergey Poznyakoff1
2018-05-30Avoid unnecessary memory reallocations during cachingSergey Poznyakoff1
* src/findkey.c (_gdbm_read_entry): Reallocate cache_entry->ca_data.dptr only if necessary. * src/gdbmdefs.h (data_cache_elem): New member: dsize * src/gdbmdelete.c: Don't free cache_entry->ca_data.dptr * src/gdbmopen.c (_gdbm_init_cache): Initialize ca_data.dsize
2018-05-27Additional validation of avail_table.Sergey Poznyakoff1
Verify that avail_table is sorted by size and that each element's size falls within allowed range. * src/bucket.c (gdbm_get_bucket): Fix bucket validation. Validate bucket_avail. (_gdbm_split_bucket): Check return from _gdbm_free. * src/falloc.c (adjust_bucket_avail,_gdbm_free): Return error code. All uses updated. (pop_avail_block): Fix eventual memory leak. Use gdbm_avail_block_validate. * src/gdbmdefs.h (gdbm_avail_table_valid_p): Change signature. * src/gdbmopen.c (gdbm_avail_table_valid_p): Traverse the array verifying address and size of each element. (gdbm_avail_block_validate) (gdbm_bucket_avail_table_validate): New functions. (validate_header): Remove call to gdbm_avail_block_valid_p. Avail_block is validated later, after it's been loaded. Bail out if header->next_block does not equal the file size. (gdbm_fd_open): Validate avail_block. * src/gdbmstore.c (_gdbm_store): Check return from _gdbm_free. Avoid endless loop in case of inconsistent h_table. * src/gdbmtool.c (_gdbm_print_avail_list): Use gdbm_avail_block_validate. * src/proto.h: Update. * tests/gtload.c: Improve error diagnostics.
2018-01-01Happy GNU YearSergey Poznyakoff1
2017-01-02Happy GNU YearSergey Poznyakoff1
2016-07-25Improve debugging and error tracing.Sergey Poznyakoff1
* src/gdbmdefs.h (GDBM_SET_ERRNO) (GDBM_SET_ERRNO2): New macros. * src/base64.c: Use new macros to set error state. * src/bucket.c: Likewise. * src/falloc.c: Likewise. * src/findkey.c: Likewise. * src/gdbm_load.c: Likewise. * src/gdbmdelete.c: Likewise. * src/gdbmdump.c: Likewise. * src/gdbmexp.c: Likewise. * src/gdbmfetch.c: Likewise. * src/gdbmimp.c: Likewise. * src/gdbmload.c: Likewise. * src/gdbmopen.c: Likewise. * src/gdbmseq.c: Likewise. * src/gdbmsetopt.c: Likewise. * src/gdbmstore.c: Likewise. * src/gdbmtool.c: Fix preprocessor conditional.
2016-07-09Don't bail out on fatal errors, unless the user defines the fatal_err functionSergey Poznyakoff1
* src/bucket.c (_gdbm_get_bucket) (_gdbm_split_bucket, _gdbm_write_bucket): Return error code. All callers updated. * src/proto.h (_gdbm_get_bucket) (_gdbm_split_bucket, _gdbm_write_bucket): Update declarations * src/falloc.c (push_avail_block) (pop_avail_block): Return error code. All callers updated. * src/update.c (_gdbm_fatal): Exit only if the user defined fatal_err function. * src/gdbmerrno.c (gdbm_needs_recovery): New function. * src/gdbm.h.in (gdbm_needs_recovery): New proto. * compat/dbminit.c: Set gdbm_errno on fatal errors. * compat/dbmopen.c: Likewise. * src/findkey.c: Likewise. * src/gdbm_load.c: Likewise. * src/gdbmcount.c: Likewise. * src/gdbmdefs.h: Likewise. * src/gdbmdelete.c: Likewise. * src/gdbmdump.c: Likewise. * src/gdbmexists.c: Likewise. * src/gdbmexp.c: Likewise. * src/gdbmfetch.c: Likewise. * src/gdbmimp.c: Likewise. * src/gdbmload.c: Likewise. * src/gdbmopen.c: Likewise. * src/gdbmreorg.c: Likewise. * src/gdbmseq.c: Likewise. * src/gdbmsetopt.c: Likewise. * src/gdbmstore.c: Likewise. * src/gdbmsync.c: Likewise. * src/mmap.c: Likewise.
2016-07-09All gdbm functions return immediately if the DB is in inconsistent sate ↵Sergey Poznyakoff1
(needs recovery). * src/gdbm.h.in (GDBM_NEED_RECOVERY): New error code. * src/gdbmdefs.h (GDBM_ASSERT_CONSISTENCY): New macro. * src/gdbmerrno.c: Update. * src/gdbmopen.c (gdbm_open): Initialize need_recovery and last_error. * src/gdbmcount.c (gdbm_count): Return immediately if the database needs recovery. * src/gdbmdelete.c (gdbm_delete): Likewise. * src/gdbmdump.c (gdbm_dump_to_file, gdbm_dump): Likewise. * src/gdbmexists.c (gdbm_exists): Likewise. * src/gdbmexp.c (gdbm_export_to_file): Likewise. * src/gdbmfetch.c (gdbm_fetch): Likewise. * src/gdbmimp.c (gdbm_import_from_file): Likewise. * src/gdbmreorg.c (gdbm_reorganize): Likewise. * src/gdbmseq.c (gdbm_firstkey): Likewise. * src/gdbmsetopt.c (gdbm_nextkey): Likewise. * src/gdbmstore.c (gdbm_store): Likewise. * src/gdbmsync.c (gdbm_sync): Likewise.
2016-07-09Per-database error state.Sergey Poznyakoff1
Last error code is stored in the database file structure as well as in the global gdbm_errno. Special functions are provided for retrieving and clearing the last error state. * src/gdbmdefs.h (gdbm_file_info): New member: last_error * src/gdbm.h.in (gdbm_last_errno, gdbm_set_errno) (gdbm_clear_error): New protos. * src/gdbmerrno.c (gdbm_last_errno, gdbm_set_errno) (gdbm_clear_error): New functions * NEWS: Update. * compat/dbminit.c: Use gdbm_set_errno to set error state. * compat/dbmopen.c: Likewise. * src/bucket.c: Likewise. * src/findkey.c: Likewise. * src/gdbm_load.c: Likewise. * src/gdbmcount.c: Likewise. * src/gdbmdelete.c: Likewise. * src/gdbmdump.c: Likewise. * src/gdbmexists.c: Likewise. * src/gdbmexp.c: Likewise. * src/gdbmfetch.c: Likewise. * src/gdbmimp.c: Likewise. * src/gdbmload.c: Likewise. * src/gdbmopen.c: Likewise. * src/gdbmreorg.c: Likewise. * src/gdbmseq.c: Likewise. * src/gdbmsetopt.c: Likewise. * src/gdbmstore.c: Likewise. * src/gdbmsync.c: Likewise. * src/mmap.c: Likewise.
2016-07-08Improve _gdbm_findkey interface.Sergey Poznyakoff1
* doc/gdbm.texi: Document gdbm_exists in detail. * src/findkey.c (_gdbm_findkey): return parameters can be NULL. * src/gdbmexists.c (gdbm_exists): Remove unnecessary local variables. Reset gdbm_errno to 0 if the key was not found. * src/gdbmdelete.c (gdbm_delete): Remove unnecessary local variables. * src/gdbmfetch.c: Likewise. * src/gdbmseq.c: Likewise. * src/gdbmstore.c: Likewise.
2016-07-08Fix error handling in gdbm_fetch, gdbm_firstkey, and gdbm_nextkey.Sergey Poznyakoff1
* src/gdbmfetch.c: Hanlde out of memory error. * src/findkey.c: Set gdbm_errno to GDBM_ITEM_NOT_FOUND if nothing was found. * src/gdbmdelete.c: Don't set gdbm_errno after _gdbm_findkey returns -1. It's been done already. * src/gdbmexp.c (gdbm_export_to_file): Return -1 if gdbm_fetch fails. * src/gdbmseq.c (get_next_key): Set gdbm_errno to GDBM_ITEM_NOT_FOUND if there's no next key. Don't call _gdbm_fatal on out of memory condition. (gdbm_nextkey): Set gdbm_errno to GDBM_ITEM_NOT_FOUND if there's no next key. * src/gdbmtool.c (fetch_handler) (firstkey_handler,nextkey_handler): Check gdbm_errno. * src/gdbmstore.c: Handle error return from _gdbm_findkey. * tests/gtdump.c: Likewise. * tests/gtfetch.c: Likewise. * doc/gdbm.texi: Document changes. * doc/gdbm.3: Likewise. * configure.ac: Version 1.12.90 * NEWS: Update. * .gitignore: Update.
2013-12-25Version 1.11Sergey Poznyakoff1
2013-12-25Minor changes.Sergey Poznyakoff1
* src/gdbm_load.c: New options: --mmap, --cache-size and --block-size. * doc/gdbm.texinfo: Document new gdbm_load options. * doc/gdbm_load.1: Likewise. * src/gdbmdelete.c: Stylistic changes. * src/gdbmstore.c: Likewise.
2011-08-03Update copyright headers.Sergey Poznyakoff1
2011-08-01Use GDBM_FILE instead of gdbm_file_info *.Sergey Poznyakoff1
2008-11-21Initial revisionJason Downs1

Return to:

Send suggestions and report system problems to the System administrator.