diff options
| author | Costa Tsaousis <costa@netdata.cloud> | 2023-01-17 19:35:14 +0200 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2023-01-17 19:35:14 +0200 |
| commit | 7279dd092c23fbafcd7edb8ef7f3f79e1a0e5ecb (patch) | |
| tree | 730e1cc845613b1751ad79f5d8ba6bdce1ad9491 /database/engine/pdc.c | |
| parent | 6be264d62788b1b50109dc1f2a0cb6f622cfb804 (diff) | |
DBENGINE v2 - improvements part 3 (#14269)
* reduce journal v2 shared memory using madvise() - not integrated yet
* working attempt to minimize dbengine shared memory
* never call willneed - let the kernel decide which parts of each file are really needed
* journal files get MADV_RANDOM
* dont call MADV_DONTNEED too frequently
* madvise() is always called with the journal unlocked but referenced
* call madvise() even less frequently
* added chart for monitoring database events
* turn batch mode on under critical conditions
* max size to evict is 1/4 of the max
* fix max size to evict calculation
* use dbengine_page/extent_alloc/free to pages and extents allocations, tracking also the size of these allocations at free time
* fix calculation for batch evictions
* allow main and open cache to have as many evictors as needed
* control inline evictors for each cache; report different levels of cache pressure on every cache evaluation
* more inline evictors for extent cache
* bypass max inline evictors above critical level
* current cache usage has to be taken
* re-arrange items in journafile
* updated docs - work in progress
* more docs work
* more docs work
* Map / unmap journal file
* draw.io diagram for dbengine operations
* updated dbengine diagram
* updated docs
* journal files v2 now get mapped and unmapped as needed
* unmap journal v2 immediately when getting retention
* mmap and munmap do not block queries evaluating journal files v2
* have only one unmap function
Co-authored-by: Stelios Fragkakis <52996999+stelfrag@users.noreply.github.com>
Diffstat (limited to 'database/engine/pdc.c')
| -rw-r--r-- | database/engine/pdc.c | 8 |
1 files changed, 4 insertions, 4 deletions
diff --git a/database/engine/pdc.c b/database/engine/pdc.c index 240d6d35aa..ae86af73aa 100644 --- a/database/engine/pdc.c +++ b/database/engine/pdc.c @@ -1078,7 +1078,7 @@ static bool epdl_populate_pages_from_extent_data( if(worker) worker_is_busy(UV_EVENT_PAGE_POPULATION); - void *page_data = dbengine_page_alloc(ctx, vd.page_length); + void *page_data = dbengine_page_alloc(vd.page_length); if (unlikely(!vd.data_on_disk_valid)) { fill_page_with_nulls(page_data, vd.page_length, vd.type); @@ -1120,7 +1120,7 @@ static bool epdl_populate_pages_from_extent_data( bool added = true; PGC_PAGE *page = pgc_page_add_and_acquire(main_cache, page_entry, &added); if (false == added) { - dbengine_page_free(page_data); + dbengine_page_free(page_data, vd.page_length); stats_cache_hit_while_inserting++; stats_data_from_main_cache++; } @@ -1227,7 +1227,7 @@ void epdl_find_extent_and_populate_pages(struct rrdengine_instance *ctx, EPDL *e if(mmap_data != MAP_FAILED) { extent_compressed_data = mmap_data + (epdl->extent_offset - map_start); - void *copied_extent_compressed_data = mallocz(epdl->extent_size); + void *copied_extent_compressed_data = dbengine_extent_alloc(epdl->extent_size); memcpy(copied_extent_compressed_data, extent_compressed_data, epdl->extent_size); int ret = munmap(mmap_data, length); @@ -1249,7 +1249,7 @@ void epdl_find_extent_and_populate_pages(struct rrdengine_instance *ctx, EPDL *e }, &added); if (!added) { - freez(copied_extent_compressed_data); + dbengine_extent_free(copied_extent_compressed_data, epdl->extent_size); internal_fatal(epdl->extent_size != pgc_page_data_size(extent_cache, extent_cache_page), "DBENGINE: cache size does not match the expected size"); } |