| Age | Commit message (Collapse) | Author | Files |
|---|
|
|
|
* src/bucket.c (_gdbm_cache_init): Accept sizes greater than the
directory size.
* src/gdbmsetopt.c (setopt_gdbm_getdbformat): Return 0 on success.
|
|
|
|
* src/cachetree.c: Remove.
* src/Makefile.am: Remove cachetree.c
* doc/gdbm.texi: Document the changes.
* src/bucket.c (cache_tab_lookup_slot)
(cache_tab_resize): New function.
(cache_elem_new): Initialize ca_coll.
(cache_elem_free, cache_lookup)
(_gdbm_cache_init,_gdbm_cache_free): Rewrite with hash-based cache lookup.
(_gdbm_fetch_data): Remove unused function.
* src/gdbm.h.in (GDBM_GETDBFORMAT, GDBM_GETDIRDEPTH)
(GDBM_GETBUCKETSIZE, GDBM_GETCACHEAUTO, GDBM_SETCACHEAUTO): New option codes.
* src/gdbmdefs.h (cache_node): Remove.
(cache_elem): Remove ca_node. Add ca_coll (collision resolution pointer).
(gdbm_file_info): New members: cache_auto, cache_bits, cache.
* src/gdbmopen.c (gdbm_fd_open): Change cache initialization.
* src/gdbmsetopt.c (GDBM_GETDBFORMAT,GDBM_GETDIRDEPTH)
(GDBM_GETBUCKETSIZE,GDBM_GETCACHEAUTO)
(GDBM_SETCACHEAUTO): Implement new options.
(setopt_gdbm_getflags): Reflect the state of GDBM_CLOEXEC and GDBM_NUMSYNC.
* src/proto.h (_gdbm_fetch_data,_gdbm_cache_tree_alloc)
(_gdbm_cache_tree_destroy,_gdbm_cache_tree_delete)
(_gdbm_cache_tree_lookup): Remove protos.
* src/recover.c (_gdbm_finish_transfer): Restore original cache settings.
* tests/Makefile.am: Add new test.
* tests/testsuite.at: Likewise.
* tests/gtcacheopt.c: New file.
* tests/setopt02.at: New test case.
|
|
* doc/gdbm.texi: Update.
* src/gdbm.h.in (GDBM_ILLEGAL_DATA): Rename to GDBM_MALFORMED_DATA.
(GDBM_OPT_ILLEGAL): Rename to GDBM_OPT_BADVAL.
All uses changed.
Old constants retained for backward compatibility.
|
|
|
|
The commit 4fb2326a4a introduced pre-reading of memory mapped
regions. While speeding up searches, it has a negative impact
on write operatons, since every remapping effectively re-reads
the entire database. See https://github.com/Perl/perl5/issues/18884
for details.
* NEWS: Document changes.
* doc/gdbm.texi: Document the GDBM_PREREAD flag.
* src/gdbm.h.in (GDBM_PREREAD): New flag.
* src/gdbmdefs.h (gdbm_file_info): New member: mmap_preread.
* src/gdbmopen.c (gdbm_fd_open): Set mmap_preread if requested.
* src/gdbmsetopt.c (setopt_gdbm_getflags): Report GDBM_PREREAD
flag, if dbf->mmap_preread is set.
* src/mmap.c (_gdbm_internal_remap): Use pre-fault reading only
if dbf->mmap_preread is set.
|
|
|
|
* src/bucket.c (_gdbm_cache_init): Minimum cache capacity is 1.
* src/cachetree.c (_gdbm_cache_tree_lookup): Fix the description.
(_gdbm_cache_tree_destroy): Speed up the tree traversal.
* src/gdbmsetopt.c (setopt_gdbm_setcachesize): Don't enforce
minimal cache capacity.
|
|
|
|
The new bucket cache uses the least recently used replacement
policy (instead of the least recently read, implemented previously).
It also allows for quick bucket lookups by the corresponding
disk address. To this effect the cache entries form a red-black
tree sorted by bucket address.
Additionally, data buckets are also cached.
* README: Describe the new branch.
* src/bucket.c: Rewrite cache support.
* src/cachetree.c: New file.
* src/Makefile.am: Add new file.
* src/findkey.c (_gdbm_read_entry): Use _gdbm_fetch_data.
This ensures data pages are cached as well as buckets.
* src/gdbm.h.in (GDBM_BUCKET_CACHE_CORRUPTED): New error code.
(gdbm_cache_stat): New struct.
(gdbm_get_cache_stats): New proto.
* src/gdbmclose.c (gdbm_close): Call _gdbm_cache_free to dispose
of the cache.
* src/gdbmdefs.h (cache_elem_color): New data type.
(cache_elem): New members: ca_left, ca_right, ca_node, and
ca_hits.
(cache_tree): New typedef.
(gdbm_file_info): Remove bucket_cache and last_read.
New fields: cache_num, cache_tree, cache_mru, cache_lru,
cache_avail, cache_access_count.
* src/gdbmerrno.c: Handle GDBM_BUCKET_CACHE_CORRUPTED.
* src/gdbmopen.c (gdbm_fd_open): Change cache initialization.
(_gdbm_init_cache, _gdbm_cache_entry_invalidate: Remove.
* src/gdbmsetopt.c (setopt_gdbm_setcachesize): Cache can be
re-initialized on the fly.
* src/gdbmtool.c: Change bucket printing routines.
* src/proto.h (_gdbm_read_bucket_at): Remove.
(_gdbm_fetch_data,_gdbm_cache_init,_gdbm_cache_free)
(_gdbm_cache_flush,_gdbm_cache_elem_new)
(_gdbm_cache_tree_alloc,_gdbm_cache_tree_destroy)
(_gdbm_cache_tree_delete,_gdbm_rbt_remove_node)
(_gdbm_cache_tree_lookup): New protos.
(_gdbm_init_cache,_gdbm_cache_entry_invalidate): Remove.
* src/recover.c (_gdbm_finish_transfer): Adapt to the new
cache structure.
* src/update.c: Likewise.
* tests/setopt00.at: Fix second GDBM_SETCACHESIZE test.
|
|
|
|
* configure.ac: Version 1.5
* NEWS: Update.
* src/Makefile.am (VI_CURRENT): Increment to 6
* src/bucket.c (_gdbm_get_bucket): Fix the upper limit for
bucket_bits.
* src/falloc.c (push_avail_block): Fill the allocated block
with 0s.
|
|
* src/mmap.c (_gdbm_file_extend): Move to ...
* src/fullio.c: ... here
* src/proto.h (_gdbm_file_extend): New proto.
* src/gdbmopen.c (gdbm_fd_open): Make sure the file size equals
header->next_block.
* src/gdbmsetopt.c [!HAVE_MMAP]: Don't disable GDBM_GETFLAGS.
* src/update.c (_gdbm_end_update): Make sure disk file ends on
header->next_block.
|
|
Rename: __read to gdbm_file_read
__write to gdbm_file_write
__lseek to gdbm_file_seek
__fsync to gdbm_file_sync
|
|
|
|
* src/gdbmsetopt.c: Ignore NULL entries
|
|
|
|
* src/gdbmdefs.h (GDBM_SET_ERRNO)
(GDBM_SET_ERRNO2): New macros.
* src/base64.c: Use new macros to set error state.
* src/bucket.c: Likewise.
* src/falloc.c: Likewise.
* src/findkey.c: Likewise.
* src/gdbm_load.c: Likewise.
* src/gdbmdelete.c: Likewise.
* src/gdbmdump.c: Likewise.
* src/gdbmexp.c: Likewise.
* src/gdbmfetch.c: Likewise.
* src/gdbmimp.c: Likewise.
* src/gdbmload.c: Likewise.
* src/gdbmopen.c: Likewise.
* src/gdbmseq.c: Likewise.
* src/gdbmsetopt.c: Likewise.
* src/gdbmstore.c: Likewise.
* src/gdbmtool.c: Fix preprocessor conditional.
|
|
* src/gdbm.h.in (GDBM_GETBLOCKSIZE): New option.
* src/gdbmcount.c (gdbm_count): Fix memory leak on
error.
* src/gdbmsetopt.c (gdbm_setopt): Rewrite.
Handle GDBM_GETBLOCKSIZE.
* NEWS: Document GDBM_GETBLOCKSIZE
* doc/gdbm.texi: Likewise.
* tests/gtload.c: New options -bsexact and -verbose.
* tests/Makefile.am: Add new testcases.
* tests/testsuite.at: Likewise.
* tests/blocksize00.at: New testcase.
* tests/blocksize01.at: Likewise.
* tests/blocksize02.at: Likewise.
|
|
* src/bucket.c (_gdbm_get_bucket)
(_gdbm_split_bucket, _gdbm_write_bucket): Return error code. All callers
updated.
* src/proto.h (_gdbm_get_bucket)
(_gdbm_split_bucket, _gdbm_write_bucket): Update declarations
* src/falloc.c (push_avail_block)
(pop_avail_block): Return error code. All callers
updated.
* src/update.c (_gdbm_fatal): Exit only if the user defined
fatal_err function.
* src/gdbmerrno.c (gdbm_needs_recovery): New function.
* src/gdbm.h.in (gdbm_needs_recovery): New proto.
* compat/dbminit.c: Set gdbm_errno on fatal errors.
* compat/dbmopen.c: Likewise.
* src/findkey.c: Likewise.
* src/gdbm_load.c: Likewise.
* src/gdbmcount.c: Likewise.
* src/gdbmdefs.h: Likewise.
* src/gdbmdelete.c: Likewise.
* src/gdbmdump.c: Likewise.
* src/gdbmexists.c: Likewise.
* src/gdbmexp.c: Likewise.
* src/gdbmfetch.c: Likewise.
* src/gdbmimp.c: Likewise.
* src/gdbmload.c: Likewise.
* src/gdbmopen.c: Likewise.
* src/gdbmreorg.c: Likewise.
* src/gdbmseq.c: Likewise.
* src/gdbmsetopt.c: Likewise.
* src/gdbmstore.c: Likewise.
* src/gdbmsync.c: Likewise.
* src/mmap.c: Likewise.
|
|
(needs recovery).
* src/gdbm.h.in (GDBM_NEED_RECOVERY): New error code.
* src/gdbmdefs.h (GDBM_ASSERT_CONSISTENCY): New macro.
* src/gdbmerrno.c: Update.
* src/gdbmopen.c (gdbm_open): Initialize need_recovery
and last_error.
* src/gdbmcount.c (gdbm_count): Return immediately if the
database needs recovery.
* src/gdbmdelete.c (gdbm_delete): Likewise.
* src/gdbmdump.c (gdbm_dump_to_file, gdbm_dump): Likewise.
* src/gdbmexists.c (gdbm_exists): Likewise.
* src/gdbmexp.c (gdbm_export_to_file): Likewise.
* src/gdbmfetch.c (gdbm_fetch): Likewise.
* src/gdbmimp.c (gdbm_import_from_file): Likewise.
* src/gdbmreorg.c (gdbm_reorganize): Likewise.
* src/gdbmseq.c (gdbm_firstkey): Likewise.
* src/gdbmsetopt.c (gdbm_nextkey): Likewise.
* src/gdbmstore.c (gdbm_store): Likewise.
* src/gdbmsync.c (gdbm_sync): Likewise.
|
|
Last error code is stored in the database file structure as
well as in the global gdbm_errno. Special functions are provided
for retrieving and clearing the last error state.
* src/gdbmdefs.h (gdbm_file_info): New member: last_error
* src/gdbm.h.in (gdbm_last_errno, gdbm_set_errno)
(gdbm_clear_error): New protos.
* src/gdbmerrno.c (gdbm_last_errno, gdbm_set_errno)
(gdbm_clear_error): New functions
* NEWS: Update.
* compat/dbminit.c: Use gdbm_set_errno to set error state.
* compat/dbmopen.c: Likewise.
* src/bucket.c: Likewise.
* src/findkey.c: Likewise.
* src/gdbm_load.c: Likewise.
* src/gdbmcount.c: Likewise.
* src/gdbmdelete.c: Likewise.
* src/gdbmdump.c: Likewise.
* src/gdbmexists.c: Likewise.
* src/gdbmexp.c: Likewise.
* src/gdbmfetch.c: Likewise.
* src/gdbmimp.c: Likewise.
* src/gdbmload.c: Likewise.
* src/gdbmopen.c: Likewise.
* src/gdbmreorg.c: Likewise.
* src/gdbmseq.c: Likewise.
* src/gdbmsetopt.c: Likewise.
* src/gdbmstore.c: Likewise.
* src/gdbmsync.c: Likewise.
* src/mmap.c: Likewise.
|
|
|
|
* src/gdbm.h.in (GDBM_GETDBNAME): New option.
* src/gdbmsetopt.c (gdbm_setopt): Handle GDBM_GETDBNAME.
* doc/gdbm.texinfo (Options): Document GDBM_GETDBNAME.
|
|
|
|
* src/gdbm.h.in (GDBM_OPENMASK): New constant.
* src/gdbmconst.h: Remove constants duplicated in
gdbm.h.in.
* src/gdbmdefs.h (gdbm_file_info) <allow_mmap>
<mmap_inited>: Replace with a single member: memory_mapping.
All callers updated.
(GDBM_SETCACHESIZE,GDBM_SETSYNCMODE)
(GDBM_SETCENTFREE,GDBM_SETCOALESCEBLKS): New option names.
(GDBM_SETMMAP,GDBM_GETMMAP,GDBM_GETFLAGS)
(GDBM_GETCACHESIZE,GDBM_GETSYNCMODE,GDBM_GETCENTFREE)
(GDBM_GETCOALESCEBLKS,GDBM_GETMAXMAPSIZE): New option codes.
* src/gdbmsetopt.c: Implement new options.
* doc/gdbm.texinfo: Document new options.
|
|
The new code is more flexible and performs better when
lots of inserts are being made (e.g. when populating the
database with new data).
* src/gdbm.h.in (GDBM_SETMAXMAPSIZE): New constant.
* src/gdbmconst.h (SIZE_T_MAX): New define.
* src/gdbmdefs.h (gdbm_file_info) <cache_size>: Change type
to size_t.
<mmap_inited,mapped_size_max>: New member.
<mapped_remap>: Remove.
* src/gdbmopen.c: Fix a typo.
(gdbm_open): Initialize new members.
(_gdbm_init_cache): Second argument is size_t.
* src/gdbmsetopt.c (gdbm_setopt): Optval argument is void*.
Handle GDBM_SETMAXMAPSIZE.
Improve error checking.
* src/mmap.c (_GDBM_IN_MAPPED_REGION_P): Fix comparison with
the lower bound.
(_GDBM_NEED_REMAP): Return true if mapped_region is NULL.
(SUM_FILE_SIZE): Rewrite.
(_gdbm_mapped_unmap): Don't call msync.
(_gdbm_internal_remap): Take 2 arguments, the second one
giving the new mapped size.
Unmap the region prior to remapping it.
Always pass NULL as the argument to mmap.
(_gdbm_mapped_remap): Rewrite the logic. Change semantics of the
third argument. All uses updated.
(_gdbm_mapped_init): Reflect the above changes.
(_gdbm_mapped_read,_gdbm_mapped_write): Use mmap_inited to decide
whether to use mmap, because mapped_region can be reset to zero
by another functions (namely, _gdbm_mapped_lseek).
Reset mmap_inited to FALSE, if _gdbm_mapped_remap fails.
(_gdbm_mapped_lseek): Rewrite offset computations. Invalidate
the mapped region.
* src/proto.h (_gdbm_init_cache): Change prototype.
* src/update.c (write_header, _gdbm_end_update): Remove checks
for dbf->mapped_region.
* tests/gtload.c: Implement the -maxmap option (set maximal
mapped memory size).
* doc/gdbm.texinfo: Document GDBM_SETMAXMAPSIZE.
|
|
|
|
|
|
|
|
|
|
|
